1 / 11
Polymorph developed from an investigation into multisensory, distributed AI operating across heterogeneous material substrates. The project assembled a complex computational ecosystem in which physical and synthetic processes co-evolve without a fixed global target. It asks what occurs when generative models participate in a system whose overall behaviour is not governed by a singular objective and whose operational relationships can reorganise through feedback and continuous learning. Transmodal transformation functions here as a distributed way of seeing: one modality registers consequences produced in another as activity passes among image, sound, vibration, sensor data, code, and physical matter.
Because these domains have different structures, rhythms, and capacities, every passage produces gains, losses, distortions, and residues. These discrepancies make representational friction perceptible. Misalignments between models, signals, and material conditions, together with noise and interference, drive continuous transformations. The project rejects fixed oppositions between signal and noise, and between pattern and randomness, attending to how interference in one domain can become structure in another, while stable patterns can generate variation when translated into a different form. Their shifting roles contribute to a complex, self-assembling, learning computational ecology.
The project originated at the Laboratory for Artificial Intelligence in Design (AiDLab), jointly established by The Hong Kong Polytechnic University and the Royal College of Art. Early research drew on forms of nonhuman perception, distributed coordination, and morphogenetic pattern formation, including cephalopod chemotactility and distributed sensing, murmurations of birds, insect swarms, and fish schools. The research examined how localised sensing can produce collective transformation across scales, and how higher-order configurations can in turn alter the conditions of local response. Cross-scale cohesion describes the temporary coordination through which processes operating at different spatial, temporal, and computational scales acquire a shared tendency while retaining their heterogeneity. This circulation across scales also makes the boundary between the system and its environment porous, as environmental signals enter its processing and feedback cycles. These principles informed the design of a computational learning infrastructure in which system-level behaviour arises through interactions among components, sensory domains, and incoming environmental data.
In Polymorph, continuous retraining allows outputs to become conditions for subsequent processes, repeatedly exposing the system to the altered consequences of its own activity. The project understands this as curious modelling: the formation of provisional models through ongoing encounters with changing conditions. Complexity science provides the conceptual tools for understanding the resulting dynamics. Emergence describes collective patterns that cannot be inferred from individual components; phase change identifies qualitative shifts in the system’s organisation; and multistability describes its capacity to sustain several possible organisations under comparable conditions. When changes in texture, rhythm, movement, and sonic density cohere across modalities and scales, these transitions become perceptible as temporarily sustained systemic moods.
Each modality therefore produces a partial account of the system, while differences between these accounts carry information about its transformations. The project’s engagement with non-reductive logic and systemic non-closure treats coherence as provisional and susceptible to further reorganisation. Within the context of computer vision, Polymorph approaches perceptibility as a transmodal, multiscale process through which a computational ecology develops provisional ways of sensing and modelling its own activity. https://polymorph.bernac.org
Polymorph is a recursive multimodal system in which physical sensing, generative inference, audiovisual output, and iterative model training are coupled through TouchDesigner. Its components include a fine-tuned Stable Diffusion model, a RAVE audio model, two steel plates, a suspended conductive steel thread, a curved metal projection surface, cameras, a microphone, speakers, surface resonators, projectors, and a monitoring display. The installation operates through rapid sensor-feedback loops and a slower model-training cycle.
Live input comes from vibration, electrical variation, camera imagery, and sound. The steel plates operate bidirectionally, registering vibration as sensing surfaces and radiating sound when driven by attached surface resonators. Movement of the conductive thread relative to the metal sheet produces a continuous proximity-dependent signal. Air currents, nearby bodies, and electromagnetic interference introduce further variation. Cameras provide live imagery for optical-flow analysis, while the microphone captures the composite acoustic environment for frequency- and amplitude-based analysis.
TouchDesigner functions as the system’s real-time coordination layer. It receives the sensor channels, extracts optical and acoustic features, and retains both current values and accumulated patterns across several temporal windows. Its control structure is based on multivariate thresholds: combinations of sensor values, derived features, and recent activity must enter specified ranges simultaneously or within defined intervals before a process is activated or reconfigured. Threshold events alter signal routing, initiate model inference, modulate audiovisual parameters, and change the behaviour of procedural animations.
This produces an event-driven data flow whose sequence is generated during operation. Stochastic variation enters through diffusion-model sampling, procedural noise fields, randomised modulation within defined parameter ranges, and perturbations within the agent-based animations. Physical sensor fluctuations introduce an additional source of variability. Recursive feedback then returns generated sound and imagery to the sensing layer. Local threshold events accumulate and interact across multiple timescales, producing system-level behaviour that cannot be assigned to any single input or control rule.
The installation makes this behaviour perceptible through two distinct visual forms. Transient creature-like entities generated through the Stable Diffusion pipeline provide discrete markers of particular configurations of sensor and model activity. Continuously morphing activity maps generated in TouchDesigner register the distributed dynamics of the installation as a whole. These maps combine point-cloud structures, multi-agent boid systems, textural fields, and changes in density, clustering, drift, turbulence, repetition, and attenuation. Their organisation develops from the interaction of multiple data streams, temporal histories, threshold events, and stochastic processes.
These forms allow activity to be followed at two scales: the creatures crystallise particular states, while the maps register changes in the relations among processes. A system-level phase change is identified when several tendencies reorganise together and a different rhythm, density, responsiveness, or morphological pattern emerges. Individual threshold crossings govern local transitions, phase change is recognised as a qualitative reorganisation of their collective behaviour across the system.
The combined visual field is projected across the curved metal sheet and surrounding walls. Reflection, surface distortion, and movement in the room transform the projection before it is captured again by the cameras. Optical-flow data derived from this transformed field returns to TouchDesigner and contributes to subsequent threshold evaluations, model activations, and animations.
The acoustic loop operates in parallel. The microphone records ambient sound, speaker output, resonating metal, and activity within the room. This signal is analysed and routed through the RAVE model, whose generated audio is reproduced through the speakers and steel resonators. The emitted sound produces further acoustic and material vibration, which is registered again by the microphone and plate sensors.
Selected generated images are stored locally and incorporated into subsequent Stable Diffusion fine-tuning rounds. Updated model checkpoints are periodically redeployed within the installation. Environmental activity therefore affects generation first through the real-time control system and later through the outputs selected for retraining. The expanded training corpus contains visual traces of the sensory and material conditions under which those outputs were produced.
The project terms this process intermodal retraining. Electrical, acoustic, optical, spatial, and model-generated data repeatedly become control signals for processes operating in other modalities. Recursive fine-tuning exposes Stable Diffusion’s inherited visual regularities to outputs produced within this changing material environment, allowing those regularities to be redirected over successive cycles. The resulting architecture links local interactions, multiscale feedback, stochastic variation, and qualitative changes in collective organisation within a single operational system.
Sonia Bernac
Dr Sonia Bernaciak is a philosopher of media and science, artist and creative technologist. Drawing on complexity science, her research examines tensions between technologies of thought that organise contemporary computational infrastructures. Her practice explores the aesthetics of generative AI and self-organisation in multi-agent systems through experiments staged in 3D environments. She has delivered invited talks and exhibited her work internationally. Recent talks and projects include 'Synthetic Disobedience: Misalignment and Incommensurability in Complex Artificial Systems' (2026), 'Synthetic Mood, or the Aesthetic Question of Ethics' (2026), and 'Deviations of a Fruit Fly' (2024).
Jeremy Keenan
Dr Jeremy Keenan creates artworks using sound, motion, feedback, data, sensors and light. His current line of practice involves generative audiovisual composition, sonic spaces, latent spaces, kinetic artworks, compound physical/virtual worlds, and hybrid media environments. He is interested in working with real and imaginary signals, audible and inaudible feedback, the lateral mis/use of de/generative AI, and the emergent possibilities hidden within ubiquitous sensorial infrastructures.
Maggie Roberts
Maggie Roberts is a multimedia artist working mostly as the collaboration 0rphan Drift. Since 2018, she has explored AI through the somatic tendencies of the octopus, as a distributed, many-minded consciousness. Her works If AI Were Cephalopod, Becoming Octopus Meditations and the current 9 Brains project explore other systems of perception and proprioception, communication with alien intelligence, and human exceptionalism’s limited understanding of our relation to other life. 0rphan Drift installations, performances and speculative fictions have been exhibited nationally and internationally in gallery and museum spaces for over three decades. ISCRI was partnered by the Serpentine Gallery’s Creative AI Lab.
Johnny Golding
Prof Johnny Golding is a philosopher and poet. Their research pays homage to ‘sticky cohesions’—radical forms of attunement: political, erotic, aesthetic and experimental. Golding is Co-Head, with Jonathan Boyd, of the Radical Matter Research proto-centre: Art ⇌ Philosophy ⇌ Wild Science at the Royal College of Art. Recent work includes Re-Wilding AI, incorporating, as Principal Lead, Polymorph, and, as co-Principal Lead with Tom Simmons, Weather Spores. Seminal works include The Colour of Time (2024), Octopussy (2023–24), and ‘The Courage to Matter’ (2022). Golding is Professor of Philosophy and Fine Art at the RCA.
Bruno Klopott
Bruno Klopott has a background in electronics and information technology and works as a software engineer. His professional experience spans metrology and, more recently, medical device development. Alongside his engineering career, he pursues 2D and 3D graphic design and creative programming as an independent practice. Since the early 2010s, he has collaborated with Sonia Bernac on a range of artistic and technological projects that span scientific, technological, and creative forms of enquiry, contributing expertise in software development, electronics, digital imaging, and computational experimentation.
