WHO DANCE
The new choreography
by Rebecca Allen
with Jane Nisselson
Rebecca Allen, producer of the animation, is a designer, director and researcher at the Computer Graphics Laboratory of the New York Institute of Technology. Jane Nisselson works with and writes about computer graphics systems at the Laboratory.
Throughout history we have expressed the need to create an entity that can assume human qualities and yet surpass our limitations. We expressed this need by creating artificial figures that personified an ideal: Galatea, for example, sculpted by Pygmalion and brought to life by Aphrodite. In such figures we project a perfected image of ourselves.
In the field of computer graphics, scientists began developing methods for graphically simulating real-world objects. And, of course, before too long the computer was used to generate a realistic model of the human figure. At the New York Institute of Technology's Computer Graphics Laboratory, we have described a three-dimensional mathematical model of a female figure. Like a living person, our figure can move and bend, performing dynamic actions and gestures. She is animated with the computer.
An example of our figure's physical capabilities can be seen in her role as Saint Catherine in the dance video production of The Catherine Wheel, directed and choreographed by Twyla Tharp. While working on the piece, Twyla visited our lab and saw my animation of the female figure. She approached me with the idea of a collaboration in which this figure would represent St. Catherine. Its premiere on PBS in the spring of 1983 marked the first time human and computer figures danced together for the public.
As St. Catherine, our computer dancer represents an ideal who offers spiritual guidance in the face of human shortcomings. Her image appears early in the performance, materializing shortly after an animated image of the "Catherine wheel" (a bladed torture device named after the martyred saint). Through her movement, she suggests "dance" and becomes an inspiration to the Leader, played by Sara Rudner. Later St. Catherine returns and teaches the Leader a dance theme that recurs throughout the performance. They dance together, in synchrony, and end in an outstretched position as the Catherine wheel reappears. Computer Catherine is seen, a last time, ascending a staircase: a symbolic gesture to calmly remind Sara of her unrelenting pursuit of excellence.
The Notion of Motion
How is it possible to make a series of images not only move in a human way, but dance? The analysis of human motion requires the study of actual human movement as well as physical structure. I began using animation and the technique of rotoscoping as a way of accurately modeling human motion in a stylized environment. Beginning with real actions, I would simplify its essence. This differs from conventional cartoon animation techniques, which typically build fictional actions in an exaggerated form.
The computer's capacity to make moving images from static key poses makes it an ideal tool for animation and motion analysis. Still images containing one of the many stages of an action can be transformed into a continuous movement such as walking, running or turning.
Depicting motion realistically requires a spatial
context that is more sophisticated than the flat or two-dimensional
space used in conventional animation. For instance, an object moving
away from us appears to become smaller. A computer can calculate the
correct appearance of objects as they move in space; this realistic
perspective is important in accurately representing movement. Thus the
computer is used as a means of analyzing human motion and as a medium
in which to portray it.
Taking the First Step
For a computer-generated figure to take even one step requires an
adequate animation system. The software for our system was developed
over many years by a number of computer scientists at our laboratory.
Using high-speed mainframe computers, they gradually fine-tuned the
software into a unique system that enables us not only to move objects,
but to generate complex motion for characters with articulated joints.
Moving a figure or any of its parts in all directions gives us the
means to create an illusion of life. The way a character moves defines
emotions and personality.
St. Catherine was modeled by taking key points from
photographs of a woman's body and entering them into the computer.
Connecting these points, the computer composes rigid body parts linked
by joints and arranged in a hierarchy, or tree structure. The figure is
viewed as a wire-frame model on a real-time state-of-the-art display.
Using a joystick, I can rotate, move or scale (change the size of) the
figure or any of its parts. The model is like a marionette; the
animator is a high-tech puppeteer who controls and positions the figure
by manipulating a joystick.
This animation system can be illustrated by the
simple example of making the figure bend forward at the waist, then
return to an upright position. I begin with an image of the figure
standing upright as the first "keyframe"-the first position of this
movement (Fig. 1). Then I locate the waist joint and, using a joystick,
rotate the upper half of the body to a bending position that becomes
the second keyframe (Fig. 2). A keyframe of the figure standing upright
ends the movement. This creates a three-keyframe sequence: standing,
bending, then standing. After determining the length of time the
movement should take, I run a program that generates all the
"in-between" positions. The result is the smooth motion of a simple bow.
Using this system to generate realistic human motion
requires complicated positioning of all body parts, e.g., the head,
shoulders, elbows, wrists, fingers, into keyframes. The analysis of a
dance is an intricate procedure, and a dance choreographed by Twyla
presented an especially difficult challenge.
Learning to Dance
The choreography for the St. Catherine figure was based on the
movements designed by Twyla for her dance company. We videotaped one of
the dancers as a model of motion, and I studied individual frames of
the tape to determine the key positions of particular movements. I then
positioned each joint of the computer figure to correspond to the
dancer's key positions. Twenty-five still frames were needed to
generate each second of animation. I produced two to three keyframes
for each second of motion, and from these the computer generated the
rest.
The animated figure and the human dancer had
different limits and freedoms, so we adapted the choreography for St.
Catherine according to her technical restrictions. Twyla would show me
the moves she wanted, and we would determine if they were technically
possible. Overall, the computer dancer's movements were kept close to
the physical range of a human dancer, with some movements extended
slightly beyond human capabilities.
To make St. Catherine appear to dance naturally, we
had to consider the physical world in which she moved. Building the
illusion that the computer figure was dancing with Twyla's company
demanded that she appear on the stage, in the same space as the live
action. During live-action filming by a BBC crew, I recorded the
position of the camera in relation to the dancers and then, with our
animation system, matched my "simulated camera" to the real camera
viewpoint so that Computer Catherine would appear to be in the actual
space.
While my dancer had the freedom to move in ways that
may not be considered physically accurate, I used the "personality" or
style of this animation system to make motion that was believable but
distinctive from that of the human dancers. Part of the challenge was
finding creative ways to bring the two worlds together.
Sometimes it was the differences between the two
dancers' worlds that provided a means to unite them. A human's
movements are continuous, with each step leading directly into another.
But the computer-drawn figure's movement is only an illusion created by
a series of frames generated by the computer shown fast enough to
appear as continuous motion. Each frame is an instant in time in which
the computer dancer makes a movement. I could use the frames to change
the timing and sequence of the dance steps. When first compositing the
human and computer dancers, we discovered that Sara moved into a turn
much faster than St. Catherine. By reanimating this section, I was able
to reduce the number of frames in which the computer dancer made this
turn. The animation system allowed me to synchronize St. Catherine's
timing with Sara's so that their turns met in space.
Counterpoint
In many ways the animator and choreographer do the same thing. Both
design movements that their performers then execute. But the method of
creating a dance with human dancers is very different from that used to
make the computer figure dance.
At the filming of the performance, the stage set was
filled with the hustle and bustle of a live-action production. Twyla's
dancers were rehearsing, the film crew was busily setting up,
telephones rang and people were running out for lunch. This was very
different from the atmosphere at the lab. Initially I worked with
another researcher, Robert McDermott, to shape the data base for the
body description, but once I began the animation my environment was
isolated and internal. Typically I worked at night, when there were
long stretches of uninterrupted time. Plugged into a Walkman, with the
music for the dance blaring over my headphones, I would sit in a small,
dark room, staring at a monitor. The figure would begin to dance as I
used the joystick to "describe" to the computer which of the figure's
limbs and joints should move and in what direction.
Like the choreographer, the animator directs the
dancer's movements, but the image of the dancer must also be created.
Certainly St. Catherine's appearance distinguished her as being from a
world other than that of the rest of Twyla's company. Her figure is
composed of lines in a way that gives a random, hand-drawn quality to a
computer-generated image. I wanted to break away from a mechanically
rendered appearance and typical style of computer-generated graphics.
Also, eliminating a lot of information found in a detailed rendering
helped to focus the imagery on the figure's movements.
Through her movement, Computer Catherine implicitly
took on the form of a dancer. And as a dancer, she was able to perfect
and share the essential elements of human movement that give it its
expressive quality. But after all, in producing this animation, we were
also bringing a knowledge of human movement and expression to the
computer. Through this collaboration I was able to discover a new
interpretation of the human form.
Return to Table of Contents | Previous Article | Next Article
