Technology

How it works, for artists

RealityJ is engineered by its artist. This page explains the four systems behind the work without assuming you have ever written code, and then makes the case for why virtual reality matters in the history of contemporary art.

Start with a line. Every element in RealityJ begins as a drawing: graphite on black paper, or a finger on a phone screen. The drawing is then rebuilt as a three-dimensional object, given rules for how it grows and moves, and placed in a world that is rendered live on a headset.

The best analogy is not film or video games but printmaking or sculpture: the artist prepares a matrix and a process, and the final image is produced each time, by the process, slightly differently. The difference is that here the process also includes you.

A monitor beside the headset shows the white line drawing a visitor is making in the air

A visitor's drawing, mirrored on screen · Museo MARCO

The four systems

What the headset is doing while you draw

Hand tracking: the hand is the brush

Think of Krueger's Videoplace, or of a camera that can see your fingers.

The Meta Quest headset has small cameras facing outward. Software reads their images many times a second and reconstructs the skeleton of each hand, joint by joint, as a set of points in space. RealityJ never sees a controller; it sees your hand. A pinch becomes a pencil touching paper, an open palm becomes a push against a portal, and the trail of a fingertip through the air becomes a line that stays. Because the gesture is the interface, no instruction is needed. Visitors of all ages understand it within seconds.

Real-time rendering on a standalone headset

A painting that is repainted ninety times a second, from wherever you happen to be looking.

Unlike a film, nothing in RealityJ is pre-recorded. The world is held as geometry, light and rules, and Unreal Engine 5 draws it afresh for each eye every frame, from the exact position of your head, so that turning or leaning changes the view as it would in a room. The headset is a small computer with a mobile processor and a battery, not a workstation, so every effect has to be earned. That constraint is part of the work's aesthetic: white line on black ground is not only a drawing style but an economy of means.

Procedural animation: rules instead of frames

Closer to gardening than to filmmaking.

The cacti that grow from the ground, the portals that open, the forms that drift through the sky are not animated frame by frame. Each has rules: how fast it grows, what it responds to, how it decays. The artist sets the rules and the seeds; the world runs them live, differently every time. This is why no two sessions are alike and why the museum text calls the experience unrepeatable. It is also what makes the work a system rather than an object, which places it in the tradition of kinetic and generative art rather than of video.

AI music analysis: the world listens

Kandinsky's colour keyboard, played by a machine that hears.

While the music plays, an analysis layer measures it continuously: where the beats fall, how loud and how bright the sound is, how its texture changes. Those measurements are mapped to the rules above. A kick drum can make the ground pulse; a swell of strings can make a cactus bloom; silence can let the portals dim. The AI generates no images. Every form was drawn by hand. The machine's only job is to hear, and to translate what it hears into growth and light.

A visitor gestures with both bare hands inside the headset on a sunlit rooftop

Hands, not controllers · San Francisco

Several visitors wearing standalone headsets at once during an open-air night session

Standalone headsets, no cables · Miami

Built in Unreal Engine 5

Runs on Meta Quest 3, 3S and Pro with the Meta XR SDK

Essay

Why virtual reality matters in the history of contemporary art

Six reasons, each tied to a tendency the art of the last century already set in motion.

The frame disappears

From the panorama to the cinema screen, immersion was always framed by an edge. A headset has no edge. The oldest ambition of painting, to put the viewer inside the picture, is finally literal, and the picture plane that defined Western art for six centuries is simply gone.

The body becomes the medium

Minimalism, performance and installation art moved the work off the wall and into the viewer's body. VR completes the move: scale, depth and distance are felt, not depicted. Merleau-Ponty's insight that perception is something we do with the whole body becomes the artist's working material.

The work has duration and is unrepeatable

Like performance, a reactive VR work exists only while it is happening and never twice in the same way. Unlike performance, it can happen anywhere, for anyone, without the artist present. This is a new category: a time-based work with the availability of an object.

The artwork is a system

Generative and kinetic art from the 1960s onward proposed that the artist makes rules and the work plays itself out. VR lets those rules include the visitor. The result is relational in Bourriaud's sense, but without needing a social situation to be staged: the relation is between one person and the world the artist drew.

Distribution replaces the white cube

Lucy Lippard described the dematerialisation of the art object in the 1960s. A free download on a consumer headset is dematerialisation in practice: no shipping, no climate control, no admission. A work that museums show one visitor at a time can also be in a thousand living rooms. That does not replace the museum, as the MARCO exhibition proved, but it changes what a museum is for.

A Latin American contribution

The call to make art out of space, light, time and movement was issued in Buenos Aires in 1946, in Fontana's Manifiesto Blanco, and carried forward by Kosice, Le Parc and the kinetic artists who followed. VR gives that programme its most complete instrument. RealityJ, made by an Argentine engineer-artist and first shown in a Buenos Aires museum, keeps that lineage alive from the South.

A note of caution. Virtual reality also carries risks that a serious curatorial account should name: it isolates the viewer behind a visor, it depends on a handful of platform companies, and its files age faster than canvas. RealityJ answers the first by mirroring the experience on a screen and keeping the artist present at exhibitions; the second and third it answers only partly, by keeping the drawings on paper and the exhibitions physical. The tension is real, and it is part of what makes this a contemporary work rather than a demonstration.

The curatorial essay and FAQ Download free on Meta Quest