Co-composition stages (a) Audience initializes grain patterns, (b) Rooster performs stochastic foraging, (c) Dynamic real-time feedback loop between audience and animal, (d) Physical transduction of pecking coordinates into paper perforations, (e) Acoustic rendering via a motorized music box, (f) Accumulation of material residue visualizing the performance’s labor and temporal passage. 1 / 3
Grain as Note is a bio-tangible interaction installation that explores interspecies co-creation and decentralized authorship, aiming to dismantle traditional monolithic artist models by constructing an ecological interface shared among humans, animals, and automated machines. Grounded in traditional Chinese Wu Xing cosmology, the artwork translates the uncoerced, innate foraging behavior of a rooster into real-time pentatonic melodies. Theoretical framing draws from John Cage’s concept of indeterminacy, shifting musical composition from composition-as-planning toward composition-as-event by relinquishing temporal control and rhythmic randomness to a non-human agent. The system integrates cultural computing with materiality by instantly inscribing ephemeral biological actions onto paper tape to form permanent material fossils and physical performance residue. Within this loop, human audience members approach the work as spatial architects; they scatter five distinct grain types representing the Five Elements across a 3-meter longitudinal sensing canvas to establish a dynamic pitch grid. The audience sets structural boundaries and intervenes in real time by adding grain, building a collaborative feedback loop alongside the animal’s stochastic foraging and mechanical execution.
Technically realized as a high-precision hardware and software transduction pipeline, the installation operates through a clearly defined system architecture. The system inputs consist of the spatial pitch layout arranged by humans along the canvas's x-axis, alongside real-time coordinate data and grain classifications captured at the exact moment of a peck. System outputs encompass high-speed serial control commands, physical perforations on continuous 300gsm cardstock, a 30-note pentatonic acoustic score rendered by a motorized music box, and the physical accumulation of paper tape on the floor. Primary components include an overhead vision camera, a Python software orchestrator, an Arduino Mega 2560 microcontroller, a custom 30-note solenoid puncher powered by an eccentric cam mechanism, and an automated music box engine. The Python orchestrator uses a YOLOv8 model to track the rooster's head gestures in real time while applying localized HSV color thresholding within a dynamic Region of Interest to classify grain types corresponding to Wu Xing scales: Corn/Earth (notes 1–6), White Rice/Metal (7–12), Mung Bean/Wood (13–18), Red Bean/Fire (19–24), and Black Rice/Water (25–30). The evaluated horizontal position maps to a discrete note index (1–30) and is transmitted via Type-C serial to the Arduino, which actuates the mechanical punches with approximately 15 N of force to cleanly perforate the paper. The paper score is immediately pulled into the motorized music box to strike physical tines, seamlessly transducing biological instinct into mechanical action and acoustic performance.
Xia Liu
University of Wollongong
Xia Liu is a media artist whose work explores the intersection of biological agency. She holds a Master of Fine Arts from Alfred University and is currently pursuing a PhD in Creative Arts at the University of Wollongong. By integrating computer vision with non-human behaviors, including animal movement. She seeks to decenter the human perspective within digital systems. Treating technology as an active mediator, she translates organic patterns into poetic sensory experiences, uncovering the hidden dialogues between the natural world and the latent spaces of the machine.
Hongbin Lin
The Hong Kong University of Science and Technology (Guangzhou)
Hongbin Lin is a PhD researcher specializing in Robotics and Artificial Intelligence. His primary research interests encompass 3D Computer Vision, Embodied AI, Autonomous Driving, and Surgical Robotics. He has authored papers in premier computer vision and robotics venues including CVPR, AAAI, RSS, and IROS.
