Ground
Works

Watching and Dreaming is a body of work that enables the viewer to peer inside the “mind” of a machine to observe its perceptions, mind wanderings, and dreams. This is not a metaphorical representation of dreams, nor a technical exercise in AI such as DeepDream [1] but the realization of a computational model of dreaming informed by cognitive neuroscience. This level of description avoids biases towards Jungian and Freudian psychology that assume dreaming is exclusively human. Dreams should not be considered independently of the perceptual capacities of the dreamer, and thus comparing this model to human perceptual abilities is problematic. For the audience, these artworks function as entry-points to consider the constructed nature of perceptions and the continuity of waking, mind wandering, and dreaming. For the artist, the artworks are sites of knowledge-making; it is through the making of artistic works that the model (computational formalization) and theory (argument that situates the model in empirical knowledge) are developed. The research underlying these artworks integrates knowledge in multiple disciplinary dimensions: (a) The computational modeling of dreaming processes (Zhang 2009; Treur 2011), (b) generative and media artworks engaging with the concept of memory and dreaming (Franco 2007; Dörfelt 2011), and (c) the conception of dreaming as imagination (Nir and Tononi 2010). In this text, Watching and Dreaming (2001: A Space Odyssey) (2014) serves as an exemplar of the Watching and Dreaming body of work. The machine attempts to learn and predict Stanley Kubrick’s film 2001: A Space Odyssey through the construction of its own subjective perception that is the basis of dreaming. “Mental” images generated during perception, mind wandering, and dreaming are subjective constructions bound to the peculiarities of the machine’s way of seeing. The body of work constitutes various manifestations of the cognitive model, not attempts to communicate the model’s mechanisms.

From 2009-2011, we worked together at the Broad Institute of MIT and Harvard (Broad Institute, n.d.), building on a study (Lieberman-Aiden & Van Berkum et al., 2009) that made it possible to explore how the human genome, the DNA contained in every cell of the body, folds in 3D. At the outset of our collaboration, our approaches seemed so different as to be, perhaps, incommensurable. The scientists used tools like mathematics, computer science, and molecular biology, whereas the artistic toolkit was focused on the construction of physical objects, with a defined shape, area, and volume. Yet over time, we came to realize that all of these tools were addressing the same goal: making invisible concepts manifest as an experience intelligible to the senses. From the beginning of the project, we worked together creating drawings. Over time, our interactions evolved to become free-flowing conversations while drawing, which became a way of seeing together. Our visual experimentations grew into a body of drawings, paintings, prints, mixed-media artworks, wood blocks, a dynamic video installation, and a suspended wire sculpture (Ranganathan, 2021), and helped advance the scientific community’s understanding of how the human genome folds.