It. Estetica sintetica; Fr. Esthétique synthétique; Germ. Synthetische Ästhetik; Span. Estética sintética. “Synthetic Aesthetics” is a phrase constructed on the lexical schemes of “Informational Aesthetics” and “Synthetic Phenomenology”, the research on artificial systems that possess or specify phenomenal states. It denotes the research on the cognitive capacities required for the production and the evaluation of Art through computational modelling and simulation.
What does being able to produce artworks mean? What does knowing how to make Art consist of? Which are the features that allow classifying or evaluating artworks, and what does their cognition imply? Synthetic Aesthetics covers many computational models that address such questions according to the tenet that understanding the capacity or the knowledge X means describing (1) the function f from the data of a problem to the values of its solution given constraints on the relevant information for X, (2) the algorithm that specifies f in terms of the format assigned to the data inputs and of the operations that transform them into the solution outputs. Therefore models and algorithms of Synthetic Aesthetics are abstract descriptions of the effective procedures one would follow to solve problems in the domain of generating and evaluating artworks. The solutions must not merely meet engineering or commonsense criteria of satisfaction, because they are steps along the path of producing visual or sound artifacts for which the qualification as Art would still stand (Colton 2012). Therefore models of the class of functions to which f belongs should tally with the cognitive capacities for painting pictures, composing music, critically appraising the stages of one’s own or others’ work as well as with the motor abilities to handle artistic mediums. Models and algorithms are then tested through the implementation in automatic or autonomous agents, that is artificial systems that sense and act on their environment, whose autonomy is a function of the extent to which (1) the interaction with the environment is based on inputs over which they have no control, (2) actions are taken on the basis of their perception and goals rather than of in-built programs, (3) the operations are selected by searching the set of possible functions, that which may lead to surprising or novel results. If the results meet one or more criteria by which naive or experts subjects would judge artifacts as Art, then agents simulate human capacity, ability and knowledge to generate and evaluate Art and one can infer that models and algorithms are functionally equivalent in a qualified sense to the cognitive functions that support making Art and critical and evaluative judgment. In Synthetic Aesthetics models differ as to the cognitive architecture, that is, the cognitive capacities and the motor abilities selected as model parameters along with their functional connection, as well as to the definition of the class of f.
The Painting Fool is a model of what enables painters “doing what they do”, construed as the connection of distinct cognitive modules that allow the system to exhibit behaviours that people would regard as “independently creative” (Colton 2012). A module is a sub-system that is specialized in processing a determined kind of information, whose operations are mandatory and impenetrable to other modules or central units (Fodor 1983; Pylyshyn 1994). The Painting Fool has perceptual and motor skill modules that correspond to the capacity of making marks on canvas to represent visual scenes. They enable the system to extract visual features, to segment images into layers and to reproduce the effects (smudging, grainy textures) for varying sizes, density and distribution of brushstrokes. It has decision-making procedures to select the colour palette and the abstraction level of the representation, which correspond to the ability of choosing representational styles and giving a “non photo-realistic rendering” of visual scenes. The system also possesses a module to store knowledge on the correspondence between facial features and emotions, learned through affective computing techniques (Picard 2002), on whose basis it can generate portraits with appropriate styles that suggest distinct emotions. To endow it with a functional equivalent of imaginative capacity, the system is provided with a teaching interface by which users tell the arrangement of shapes and colors generic scenes should have. The system treats users’ clues as constraints to solve problems in painting scenes. As the functions satisfying the constraints vary, the system ends up finding different inventive solutions (Colton 2008). The teaching interface permits to train the system extensively so that it can learn and progress as a real artist unlike AARON, an earlier successful artificial intelligent system that could produce a restricted set of figurative pictures (McCorduck 1991).
Following the research on genetic algorithms that mimic biological evolution, other models describe the capacity for Art as a function of the conditions at which agents interact and vary within an environment given the selective constraints of limited resources and adaptations (McCormack 2005). In such models the class of f does not include built-in or user-defined functions to generate artifacts on the basis of one desired measure of fitness in connection to the needs or the preferences about the factors of their production or evaluation, for instance maximizing the occurrence of perceptual properties given some options for the material and its arrangement. Rather, functions are equated to genotypes whose mutation and recombination regulate the rate at which the specifying algorithms are selected, compiled and executed to generate artifacts in a manner alike to expressing phenotypes. If the artifacts show properties that qualify as pleasing or creative, the function that regulates their generation can be considered an emerging aesthetic fitness function.
An evolutionary aesthetic fitness function can drive autonomous agents like a painting machine (Anguilar et al. 2008), for which disjoint sets of painting stroke instructions are coded as distinct algorithms that act like genes in chromosomes. A scanned photograph is given as input to the system that analyzes the reflectance of pixels and simulates many digital reproductions of it. A software comparison matches the original image and the simulations and gives the latter a score depending on how many errors (differences) they show. The fitness functions that select algorithms with scores denoting intermediate amounts of difference between the extremes of the photographic reproduction and the alteration of the original image command brushstrokes instructions to a robotic arm that spread acrylic colours on a canvas accordingly. The resulting painted image resembles a kind of abstract art.
The Drawbots project is an “embodied thought experiment” about the minimal condition of creativity arising from agents behaviour in a selective environment (Bird and Stokes 2006). Agents are autonomous wheeled robots that draw lines with a pen on the field over which they are free to move. The fitness functions reward differently motor behaviours according to (1) the times drawn lines intersect with one another, (2) the coordination between lowering or rising the pen and motion, (3) the correspondence between the drawing/moving behavior and initial or acquired energy. Robots that gain more are allowed to move more and more along new paths. If pleasing drawing patterns ensue, the fitness functions capture the ability of drawing. If robots are allowed having preferences on the density of the lines drawn and spawning offspring, which replicates their behaviour, one can simulate the co-evolution of fitness functions and the environment (MacCormak and Bown 2009). As robots draw lines while moving, the resulting regions with lower or higher density of marks give them further incentives to evolve so that divergent line patterns in the environment promote the replication of agent with different preferences. If robots are endowed with a self-observation mechanism that links the inherited preferences to local characteristics of the environment, the variability of paths and consequently the “irrationality” of the drawing behaviour increase (MacCormak 2010). Magnus (2006) defines genetic algorithms for music. Algorithms operate on digitized waveforms that are played by a set of speakers with different probabilities. This variability induces errors in the algorithms that have the task of generating waveforms with some target acoustic parameters. The contrasting forces of the variability and the task causes algorithms to be recombined and mutated. The emerging fitness function generates waveforms with frequency and envelope for sounds that resemble samples of musique concrète.
In the domain of evaluating artworks, the models of Synthetic Aesthetics regard the knowledge required to classify or discriminate artworks according to their features. Machado et al. (2004, 2008) develop an “artificial art critic”, whose cognitive functions are connected through a neural network rather than a modular architecture. For pictures the system extracts image features that are entered in the computation of the local interdependence of pixels, a measure of image complexity, and of the structure repeated at coarse and fine levels of the image, a measure of perception complexity. The ratio between the two measures gives the aesthetic value of the painting. For music the system extracts pitch, octaves, melodic and harmonic intervals, their combination and the balanced pattern of their distribution at distinct levels of the composition. The system uses the extracted features to evaluate the style of paintings and music pieces with the task of identifying their author. The “artificial art critic” is able to discriminate artworks by Gauguin, Van Gogh, Picasso, Kandinsky, Goya as well as music by Purcell, Scarlatti, Debussy, Chopin with a high rate of success.
Evaluative models differ with respect to the appearance level at which the features are identified for successful discrimination. Wallraven et al. (2008) find that low level features like color or texture are not as effective at correctly sorting paintings to categories of Gothic, Renaissance, Romantic, Realist Impressionist, Expressionist, Surrealist and Post Modern Art as higher level features are. Condorovici et al. (2015), Marchenko et al. (2005) show that low level features improve the discrimination of automatic systems if specified as perceptual or artistic descriptors rather than as mere gauge of image properties. Manaris et al. (2003) apply different metrics to low and high level features of sounds to capture their correlation with aesthetically appreciated music. If the measures are used for genetic algorithms functions, an automatic system will generate similar music samples while discarding non musical samples.
Synthetic Aesthetics can be assessed under distinct respects. Do modular agents attain combinatorial, exploratory or transformational creativity (Boden 1998)? Is there a sensible difference among creative, pleasing and artistic results of evolutionary agents (Galanter 2012)? To obtain valuable patterns a pre-defined aesthetic measure may be required, which however would limit their autonomy. Those questions notwithstanding, there is room for future research if Synthetic Aesthetics is connected to the received view of Cognitive Sciences. If models and algorithms are coordinated to Aesthetic theories, and if simulations are construed as explorations of the meaning and the denotation of the concepts by which we understand Art, the qualified equivalence between human subjects and agents can give a contribution to making the assumptions of Aesthetics explicit.