Its great to talk big when it comes to the next generation of narrative games and films.
Now let’s talk smart.

1. Who owns the pixel? Digital rights and performance.

   I’ve been following the news from GDC, and of course Tameem Antoniades (Ninja Theory) and David Cage (Quantic Dream) are bound to attract my attention.  I’m not surprised by what they are saying. The general drift is, that narrative and game can work together either to deliver an immersive emotional experience that adds to the game experience and informs it, or to deliver entirely new experiences.  Their talks speak of the convergence of talents and skills that I admire, in an environment that is often hostile to new and convergent approaches.

   These are two great developers, who, in their position, are similar to powerful film directors – head of all departments with the right to the final cut.  Something that is less common than you would expect in the film industry.  However, their power over what an audience perceives as the actor’s performance will only increase.  Capture is approaching film in its ability to capture a performance, and cutting edge animation tools can now work with this material to invisibly alter every moment.

   Tameem once said at a conference (Develop 2010) that Andy Serkis (playing Monkey in “Enslaved”) delivered a performance in one scene that Tameem thought wasn’t right.  Serkis was his own director in the capture studio, but when the data came through the pipeline, Tameem decided he wanted to change the tone of Monkey’s/Serkis’ reactions and therefore the scene.  There was no budget, or perhaps time, to re-capture the scene.  So Tameem pulled an old editor’s trick (rarely possible in 2d)  by using the contextual freedom available in animation.  He pulled expressions from other scenes, or “off-cuts”, and edited them into the scene.

   In the movie, “Paul”, Arne Kaupang and his team at Double Negative often performed their own versions of the body behaviours of Paul, whilst Seth Rogan’s facial and vocal performance was kept  (From Arne’s Oslo –  ”Digital Story telling” presentation).  I’ve seen footage of animators, acting to video, for reference to animate those movements, and join them to an Actor’s head and facial behaviour.  I doubt that this is all that rare.

   In Heavy Rain, I’ve seen moments where I remember a performing a sequence of emotionally driven actions that made truthful sense to me, but (I hope for gameplay reasons) , these were re-ordered  – changing my performance and, in my opinion, the truth of the scene.

   I love acting. I believe the actors craft, executed at its best, proves itself the most transformative art-forms of interpretation. However, the moment that acting became ‘captured’ rather than ‘live’, the power and success of a performance became shared with talents beyond the moment of capture.  The highly collaborative process of Film, for example,  can be as transformative and meaningful as the best live theatre, and on some levels, even more so. I hold that it is best to consider the actor’s final performance as inviolate, but in this new era, believing this to be the only perspective is to be damagingly ‘anti-convergent’.

   Capture and animation is already pushing this balance between control of the source performance and the end ‘cut’ even further.  Capture-based animation means that not only does the developer, and sometimes the player, choose the visual focus of a scene as a director might in film, but now the developer can alter the very fabric of the performance itself.  Ownership of that final product is already becoming a point of argument and debate, and more and more people and organizations are even now being required to define their position, or risk losing their voice.

   Ownership of a performance

   

   Imagine this: a famous actor is chosen to play a part in a capture based, CG heavy film.  This requires his face be animated into that of a gorilla/fox/reptile, but his facial performance will be key-framed to be as identical as possible to the original performance.

   As it stands, the moment that an image is turned into data, the image of the performance no longer ‘belongs’ to the actor in the way it would in film. It’s an important point to the actor that his performance is respected, considered inviolate – as it is his/her name that is attached to the movie, and they will be judged and criticized for the end performance and possibly even the success of the movie.

   Where source performance and final performance can be considered identical, on what basis can it be said that the performance no longer belongs to the performer?  It’s a bit like saying that because an image creates a chemical reaction on celluloid that mirrors the image coming in through the lens, that the image now belongs to Fuji or Kodak.

   Instead, image rights have become tied to the physical appearance of an actor.  In film this is normally an unambiguous definition.  When it comes to animation, this no longer works, as the appearance can be altered, whilst the performance is retained, or vice versa, and degrees of variation in between.  Once captured, actors needn’t even be present to be used in a sequence. On Heavy Rain, “Madison” was a composite of two performers, and I myself occasionally ‘moonlighted’ as the body for another character.  I accepted this as the nature of the medium I found myself in at the time.  I do wonder if I would I accept it now…

   

   My collaboration with Dynamixyz has made it possible to create a “fingerprint” of an individual’s facial expressive behaviour.  Even with their default version (“director”), this can be used to eliminate keyframing almost entirely, enabling the animator to move smoothly from one expression to the next.  This same system forms the basis of a realtime capture system, that can achieve a very high quality result, using the “fingerprint” for reference.

   The fusion between actor and animation has now become that much deeper.  And whilst capture contracts are made ‘per-project”, the ownership issue remains relatively quiet.  However, the questions this imminent shift of control poses to my role as an actor, and as the director of my company, and as an advocate of convergence, are at the very cutting edge of the future that will face traditional and ‘new’ media alike.

   How will Actors retain rights over their “expressive behaviour”?

   If the software uses an actor’s fingerprint once, it can do so across multiple projects and platforms.  The material from one capture based movie/game could be re-used by the owning studio for other movies/games – reducing the actor’s involvement – one day, maybe entirely.  However, the prize is the democratisation of quality performances in dramatic content, more professional actors being involved in convergent media, and greater convergence between the industries as a whole.

   Perhaps it’s worth considering that one day actors and animators will share credits according to the balance between free animation and capture-faithful performance animation?

   Animators, and animation teams are often the last to be praised or awarded. This will change as animation will allow another pool of talent to manipulate a performance, requiring discretion and an understanding of acting.  Body doubles already exist in film, and Seth Rogan may or may not know who played his Paul’s body (It seems to be easier to successfully mix different bodies to heads).  But what if his facial and vocal performance were 20% animator created.  How about 50%?  I can anticipate that there will be actors who will consider this a corruption of their craft – and animators too for that matter. However, I think that this approach is one of burying your head in the sand.  Besides, what’s to say that one form of collaborative performance is better than another – as long as it’s credited as such?

   What’s coming up…

   There is a company in France that is already attempting to draw the lines of engagement – or rather provide a basis upon which to engage.  Agence de Doublures Numériques. (Digital Doubles Agency).

   Digital Doubles Agency provides a service that can assess just how much of the actor’s performance remains in the final  animation – the idea being that within a certain threshold, the actor can be re-assured that they can still put their name on the tin.

   This also suggests that there will be scope to blend an actor’s performance with the animators art, whilst being able to apportion just credit, and pay, where it’s due.

   Actors, Unions, software developers, animators, film and game studios will all have to hammer out agreements with each other to protect their interests and integrity, whilst also paving the way for new art-forms that will have convergence at their heart.  If not, then valuable talent will be excluded or wasted.  No industry should believe their worth will be accepted as a given fact, instead it may be more useful to concentrate on the worth of the possible future that can be shared, and prove the worth of our place within it.

   Posted: September 2, 2011 at 12:16 am