When Will It Be Done? Part 3

OH YEAH, STORY

I’ve talked about the Question. And about multi-plane animation. But what every filmmaker is supposed to talk about is “story.” It’s the one point about which filmmakers are supposed to have complete clarity with any project…wait for it…except this one.

In July 2004, my first idea was to create a one-act play, with two actors talking and images projected behind them. The story was a debate about space exploration concluding with a surprising reveal connecting space exploration with our fear of the unknown. That soon turned into a nine- to ten-minute film idea.

I shot that film twice in 2004 & 2005. The first time, I felt I failed in what I was trying to capture, both creatively and technically. I rewrote, reshot, re-edited the whole film but never released it as I still felt it missed the mark. I shelved the films. However they might make an appearance on the backer/donor only DVD/Blu-ray of the film 🙂

Then a few months later in early 2006, the Ferry Corsten remix of Samuel Barber’s “Adagio for Strings” came on my car stereo, and I had complete vision of a film — music plus photo-animation visuals of Saturn, space, and more, combined into a powerful emotional experience. I wrote the script for a 12- to 15-minute version aimed at film festivals, and maybe planetariums, as I was ignorant of the giant-screen and fulldome markets at that time.

My vision of the story was an artful documentary with poetic narration and staged scenes of a man watching space exploration from Earth, interspersed with photo-animations from space — a film festival piece.

This is the original concept trailer:

BACK TO MULTI-PLANE

I set about delving deep into photo-animation determined to find a method that would allow more dynamic footage and work with Saturn’s rings. It was difficult work. But I found some answers in unexpected places. I was at a play and noticed the painted background flats and how when well executed, the human vision builds depth where zero depth exists.

That led to research on moving flats locked to the movement of the camera used in motion pictures, like the forced perspective for the hobbits in Peter Jackson’s Lord of the Rings. I realized there were basic mathematical principles of movement, geometry, and perspective acting on the human visual system that could be studied and emulated.

But in order to execute this, I realized that in order to execute this with photographs, I would have to use multiple photographs carefully chosen for the right perspectives and resolutions and going far beyond the 2.5D multiplane from the Kid Stays in the Picture and other similar projects. The complexity increase was exponential but I felt it was solvable.

I began building test animations of simple objects incorporating multiple photographic layers, linked mathematically with accurate, real-world values of scale and distance. But that led quickly to another huge problem. Once I started having multiple photographs arranged into multiple planes with real-world distances, the resolution required of the photographs grew exponentially as well.

The simplest way to explain is when photographs are increasingly set at oblique angles to your camera, both physical camera or virtual computer camera, the required resolution of the plane titled close to the camera becomes many time more than the resolution at the far end of the photographs. Just pick up a photograph and look at it a 45 angle to your face. You will quickly see the problem yourself.

Almost no photograph, even the largest panoramic mosaic from NASA made from 39 Cassini photographs was enough even for a simple HD-resolution shot with a dramatically titled plane. Pouring through the Cassini photographs, I realized the raw data was there for potentially massive resolution shots to solve this issue for giant screen resolutions, but nobody had put it together.

THE RING MOSAIC, OR HOW I BECAME AN IMAGE PROCESSOR AND LEARNED TO LOVE RAW DATA

The Cassini spacecraft has taken over 350,000 photographs of Saturn to date, all black-and-white, 1-megapixel (1024 x 1024 pixels) images. When a color photograph is taken, three exposure are required with a red/blue/green (RGB) filter wheel turning in front of the camera. NASA’s Jet Propulsion Lab (JPL) programs Cassini deliberately to take some of these photographs as panoramas, exactly the same process you use with a cellphone or digital camera, moving the camera side-to-side in panorama mode. Except JPL does multiple rows as well, for a taller panorama.

But JPL’s image processors have two large challenges. First, Saturn and its moons are moving, the spacecraft is moving, and taking three photographs to get a color one means a 12-image panorama actually requires 36 photographs. The movement of camera and subjects, plus the B&W-to-color process, means automating this process is impossible. It’s 100% manual only because even the most advanced panoramic stitching software available today can’t come close to accounting for the parallax problems with camera and object movement not to mention separate color channels photographed at different time and locations.

Cassini lacks the processing power or stored energy to process images anyway, so all image processing occurs manually by people on Earth. It’s difficult work, because all sorts of corrections have to be made to account for movement and RGB color-channel alignment.

If you curious to try this yourself, here is an excellent tutorial from The Planetary Society.

It should be noted that despite the processing we are doing to create multi-plane images, we never visually alter a photograph in any meaningful way. The most alteration occurs before the image data become a final photograph, including every released photograph from Cassini, Hubble, or any other space mission.

NASA’s staff and other scientists do some image processing. As the raw data comes from the spacecraft, it’s posted online as JPEG files. Months later, better quality TIFF files are posted to NASA’s Planetary Data System, which is not for the faint of heart. But it’s free and public domain. There is a passionate global community of volunteer amateur space image processors, such as those found at the Unmanned Spaceflight Forum, who have processed images for decades, some exceeding the best scientific work.

Other space missions (Hubble, Solar Dynamics Observatory, etc.) have similar image repositories with varying amounts of public access. So I dived in and downloaded everything I could on Saturn’s rings, not just from Cassini, but from the Voyager missions and Hubble as well.

No intentional, complete panorama of Saturn’s rings was there to be found. But Cassini makes accidental panoramas by virtue of taking 350,000+ photographs over a decade plus of orbits. Because they are accidental, they are sloppy, incomplete, and profoundly challenging to work with.

It took me ten-hour days, six to seven days a week, for four months, to create a panoramic mosaic image with high enough resolution for multi-plane animation of Saturn’s rings. Over 400 photographic sources, many of them mosaics themselves, combined into one final image, obsessively built referencing other Cassini images to be sure it was accurate and complete.

THE FIRST SATURN ANIMATION

The very first animation was a B&W multi-plane shot using a suitable Saturn 1-megapixel photograph as background plane in a HD animation with my ring panoramic photograph in midground and the Saturn photograph moon layered to the foreground. Using my newly developed, mathematically based, multi-plane techniques, the animation worked! Not perfect, but it felt like a real camera photographing a scene not a Ken Burns-style pan and zoom.

But it was hard work, the software frequently crashed at 1080p resolution and getting everything accurate to photographic sources slowed the process down. So in early 2006, a year after my “vision,” I could barely eke out a 30 second B&W HD animation.

At end of January 2006, I went to Las Vegas to meet a senior member of the Cassini team at a conference where he was speaking. I was hoping for his help in getting easier access to Cassini imagery to help solve the many challenges I had run into so far. That meeting went badly, as cooperation was flatly denied. I found out later that this was through no fault of mine or my project, but that was little comfort.

Discouraged, I was sitting at a lunch table when the guy across from me introduced himself and asked me about my project. It was James Hyder, editor of LF Examiner, the trade publication of the giant-screen industry (formerly known as large-format), or what the public calls “IMAX films.” He was genuinely excited about my concept — the reaction I had hoped for at my other meeting — and told me it had to be on the giant screen.

I was humbled and complimented. But I knew the truth. I told him “it’s impossible.” Photo-animation simply could not be done at that size and resolution.

It should also be noted that James Hyder today probably now fully appreciates why ten years ago I told him “it’s impossible.”

Part 4 is here!