HOW BABYLON 5 IS TRANSFERRED TO VIDEO (C) 2001 Henrik Herranen 1. Disclaimer Everything presented here is based on what JMS (Joe M. Straczynski, series creator, scripter, producer, deity, etc...) had said over the years, combined with experimental and reverse-engineered data. Although I cannot absolutely guarantee that 100% of what is presented here is completely accurate, I believe that this presentation is essentially correct, and as a professional in signal processing and a video technique enthusiastic for 20 years, I also believe the reverse-engineered data to be correct. 2. Motivation I spent several days preparing this article with the main motivation of gatting to see Babylon 5 on DVD in as high quality a presentation form as possible. 3. Abstract This document will present how Babylon 5 is transferred from film and computer files to the video presentations available today on NTSC VHS video, broadcast and the recent DVD release of two telefilms. Emphasis will be laid on the DVD release as it has the potential to be the definitive release of the series. PAL masters will not be discussed since issues involved with making a proper PAL transfer are too complicated for the scope of this article. Although DVD has vast potential for presenting Babylon 5 in its long-awaited widescreen presentation, there are several hazards in the process, which will be discussed in this article. The main problem is that while the complete series (except The Gathering) was shot in a widescreen aspect ratio, special effects shots have not been properly prepared in order to create a high-definition widescreen presentation. Even worse, it will be shown that the current methods of making effects widescreen are far from ideal. A solution for the problems is also presented. 4. Some Video Aspect Ratio Terminology Before I go to the main article body, I must make you aware of certain aspect ratio details. The scope of these words are somewhat larger than defined here, but I'll only use the words in the sense defined here. 4:3 - The conventional NTSC aspect ratio you see on TV every day. Contains approx. 480 visible horizontal scanlines. Babylon 5's original transmissions, VHS tapes and Laserdiscs are offered in this aspect ratio. Letterbox or LBX - A way to present Babylon 5's 1.78:1 material on a regular tv. With this method a widescreen image is shrunk so that it fits to a regular 4:3 TV. An unfortunate side effect is that one fourth of the active image area is left unused because of black bars that appear on top and bottom of the picture. Thus, in the visible area you have only 360 scanlines left. SciFi Channel's Babylon 5 transmissions are offered in this aspect ratio. Anamorphic, 16:9, or Anamorphic 16:9 - A way to encode widescreen material so that there are no unused scanlines (in Babylon 5's case). The idea is that while squeezing the image horizontally so that the full width of 1.78:1 material can fit in a 4:3 TV, the same squeezing is not done vertically. Thus, if you watch a 16:9 transmission on a regular 4:3 TV set, people and things appear thin and tall. All 16:9 TV screens can show this data in the correct aspect ratio, as can 4:3 TVs with a so-called "16:9 squeeze button", a feature very common in Europe. The benefit of this approach is that you can have widescreen material and still retain all 480 scanlines. DVD players can convert this data to regular letterboxed material for people that can't do the unsqueeze in their TV sets, but in this case all the benefits of 16:9 encoding are lost, and you are back to 360 visible lines. Babylon 5 DVDs (exception: The Gathering are offered in this aspect ratio. Widescreen or WS - A Widescreen (1.78:1) image, achieved on video either by Letterboxing or by an Anamorphic transfer. 5. A Bit of History When the Babylon 5 series first started airing in January 1994, the series creator Joe M. Straczynski (JMS) told that the series is actually shot in a widescreen ratio so as to make it HDTV compatible when the time comes. Ever since then fans have waited for a chance to see Babylon 5 in its "intended" aspect ratio of 16:9 (1.78:1), or 1.85:1, or whatever the correct aspect ratio would be. Seven years after the series started, SciFi Channel started presenting Babylon 5 in letterboxed widescreen. While there were some problems with some effects incorrectly composited (remember The Teapot from MotFL?), and some whole seasons transferred incorrectly, these errors were corrected from later showings and it was generally felt that these were the definitive editions of the series - so far. Now, almost eight years since the first regular episodes of Babylon 5 aired, first telemovies The Gathering and In the Beginning have been transferred to DVD for anyone to buy. While The Gathering was always intended to be presented in 4:3 and is thus of little interest to this article (except that the transfer is proper and special effects work as they should), In the Beginning has been transferred in a so-called anamorphic 16:9 aspect ratio, which is the best way a DVD can be encoded. 6. Theory of Transferring Babylon 5 to Video 6.1 How Babylon 5 Goes from Film to Video Babylon 5 is shot on regular Super-35 film at 24 frames / second. Depending on the application, Super-35 has an aspect ratio of roughly 1.65:1. Sound is not recorded on the actual film, but on separate audio recorders. Thus, Super-35 is not a format fit for presentation, but for recording it is just fine. After a show has been shot, the Super-35 film is transferred to a digital, high-resolution format. The good thing about using a high-resolution format is that later you can use that master to scale several different resolution versions (NTSC 4:3, NTSC 16:9, PAL, etc) without noticeable quality loss. 6.2 How Babylon 5 Special Effects Go to Video Babylon 5's special effects have always been generated at NTSC resolution (approx. 720x480 pixels), 30 frames per second, for the TV aspect ratio 4:3. This holds also true for composite shots, i.e. those shots that have mixed live action and computer imagery. There are several implications of the figures described above, and they all come down to this: there is no high-resolution master for the special effects shots. Thus, if anyone wants to make a Babylon 5 transfer to any other resolution or aspect ratio than standard 4:3 NTSC, it is simply impossible to retain full video quality. The only question is how much quality will be lost. The way described above is true for Babylon 5 with the exception of five last episodes and the telemovies made after completion of the Babylon 5 series. In the Beginning was done as described. 7. Examples of Transferring Babylon 5 to Different Aspect Ratios This chapter will present you with reconstructed examples of how Babylon 5 is transferred to video. The word "reconstructed" is there because I naturally don't have access to the original, digital master files. All examples are presented vertically at approximately 1/2 of NTSC resolution. Horizontally the pictures have been scaled so that normal 4:3 images have a correct aspect ratio. Resolution on the high-resolution images are not to be taken literally. I have no idea of what resolution is used in the editing system. 7.1 Example: Live Action Material [Live Image, High Resolution] Above you can see an original, scanned Babylon 5 Super-35 frame, in an aspect ratio of 1.65:1. Because the resolution of this image greatly surpasses the resolution of NTSC (or PAL, for that matter), it is easy to choose any area and zoom into that. [Live Image, High Resolution, Boxes] This is an example of how 4:3 (blue box) and Widescreen (red box) version could be transferred from the current shot. Note that although I have chosen to make the boxes as big as possible, their size can be adjusted on a shot-by-shot basis if some details are to be emphasized. [Live Image, 4:3] In this picture we see the final 4:3 transfer. As the original high-resolution image has been scaled down, the end result still has full NTSC resolution and may look very good indeed. [Live Image, Lbx] [Live Image, 16:9] These two images show how Letterboxed and 16:9 transfers would look like. As with the 4:3 transfer, both are as sharp and clear as they can be, although the 16:9 transfer would look better with a proper 16:9 capable display device, as it has retained more information of the original high-resolution scan. 7.2 Example: Composite and Computer Generated Material In the previous example, we had the luxury of starting with a high-quality, high-resolution film frame. Unfortunately this is no longer the case with composite and CGI shots. For economical reasons they were rendered only with 4:3 NTSC resolution, with the thought that when higher resolution scans would be needed, computing power would be so cheap that regenerating the effects would be a trivial matter. Unfortunately this theory was wrong. Regenerating effects would need a lot of handcraft and is at least for now outside the range of things that can be realized. Thus, we have only original 4:3 NTSC frames to start with. [Composite Image, 4:3] As can be seen, we have much less room to manoeuver than with the live action data. How can we get a widescreen transfer out of an image that does not have any extra side information? There is only one way: cropping. Babylon 5's effects are supposedly generated in such a way that allows for matting the top and bottom of images so that you would get the correct aspect ratio. [Composite Image, Lbx] Above you can see the copped version of the 4:3 image. If, as in this case, the image is protected in such a way that no important information is lost when cropping, this is not such a bad way of making a widescreen transfer: the part of the picture that is not lost to the black bars suffers no resolution loss as it is not touched in any ways. Thus, SciFi Channel's letterbox Babylon 5 has pretty much untouched special effects, with full picture quality - unless someone has screwed up something. Finally, we'd want to make the ultimate, 16:9 transfer. And it is here we run into trouble, big time. To create the 16:9 version, we should start with the previous letterboxed image, since that's the best and only widescreen image we have available. This image that has 360 active scanlines must be converted to an image that has 480 scanlines. It is probably very easy to understand that when upconverting something from a lower resolution to a higher resolution, you cannot expect the end result to have the full resolution that the higher resolution image can present. What may not be quite as intuitive is that you cannot even get to the resolution of the lower-resolution image! What am I saying? Yes, I am actually saying that for a 16:9 letterboxed transfer, there is no way that the composite and effects shots can look even as good as in the Letterboxed version. The difference need not be big, but 16:9 will in this case necessarily be worse than letterbox. Does this sound a bit nasty? Sorry, it's worse than that. See the pictures below for the gory details. [Composite Image, 16:9 Transfer] [Composite Image, 16:9 Suggestion] Above you see two pictures that show how the 16:9 transfer could be done. The picture on the left is the way the resoution conversio has actually been done, and the picture on the right show how I would do it. As you (hopefully) see very clearly is that the picture on the left, presenting the way Babylon 5 has been transferred, looks quite bad. Some horizontal lines have been doubled, and the picture's overall look is fuzzy and unclear. You may have scaled computer bitmap pictures without ever getting as bad results as have been shown above. You might even ask yourself: "Surely nobody would be so stupid to make such an ugly transfer when it's so easy to make it properly." That's a good question. Unfortunately, in the video world, things are more complicated than this. This will be described in the next chapter. 8. Field-based Vs. Frame-based Video Transfer One nasty detail in video picture that always (including this case) comes to bite you in the ankle is that unlike film frames, video picture is transferred not in frames, but in fields. I won't go into details in why this choice was made in the late 1930's, but suffice to say that the it made it possible to transfer a HDTV signal (as NTSC was called at that time) in approximately half the bandwidth that would otherwise be required. [Interlaced Video Picture] The example picture above shows how a hypothetical, 12-line interlaced TV system works. To show the full frame, first odd scanlines are presented. After this, even scanlines are presented. If presented fast enough, these two video fields meld in the brain and form the full frame presented on the left. As a short summary: as all other video systems, also NTSC uses interlaced video. NTSC's field rate is 60 Hz (or 59.94 to the anal retentive), and the frame rate is 30 Hz. It is no coinsidence that Babylon 5's specialeffects have been generated at this same rate, 30 frames per second. It has the potential of making movement look smooth and nice compared to 24 fps live action material, which needs some tuning to fit into the 60 fields/s video system. Now that we understand these basics, we can finally see how Babylon 5's special effects are converted to 16:9. [Video Signal Path] Originally, when the effects are created, they are created at 30 fps, and stored to a computer in this format (F1, F2, F3 and F4). To transfer the images to video, they are separated into video fields. For instance, F1 is broken to f1.1 and f1.2. Likewise, F4 is broken to f4.1 and f4.2. At some point the data is letterboxed. Although it doesn't matter where this is done, I've presented the step here for clarity. Thus, we've now lost the supposedly unimportant bottom and top of the effects shot. It is worth noticing that at this point, the information of which field-pair contain data from the same source frame, is now lost. Thus, while f1.1 and f1.2 now represent the same frame, this could also be the case for f1.2 and f2.1 (not numbered in the picture). Singal path A presents how Babylon 5's transfer is done. First A1 removes the unneeded letterbox lines. Then, A2 upscales the picture by 33% vertically. This is the exact step when picture qlarity is lost. A3 regenerated interlaced fields. A4 isn't necessarily performed at all, but the combined frame F'1 is a convenient way to show how the end result will look on your screen. What went wrong with transfer A? The problem is that upscaling was done to interlaced video fields which, if taken from a signal processing view, is horrific mishandling of an innocent signal. So, how should things be done then? Signal path B shows my suggestion of how the effects should be transferred. First, F4 is broken to video fields f4.1 and f4.2 just as happened with F1. But in this case, B1 combines two video fields to create a full frame. Then, this complete frame is upconverted to create directly F'4. If needed, interlaced fields can be easily created from F'4 if that is required. You may again ask why Babylon 5 is not converted in this way which seems so much easier to do. The reason is that there is really no guarantee that the original fields are aligned. Also, Babylon 5 has several shots where live action is in 24 fps but CGI is in 30 fps, making it impossible to sync perfectly to the source signal's frame rate(s). It is worth noting that if field-combining fails, the end result will look considerably worse than any of the methods presented here. Thus, to get correct conversion, there needs to be some automatic logic combined with (only) some human interaction where automatics has guessed wrong. There are two reasons why I am so concerned with this. One is that I want to have Babylon 5 on DVD with as high quality as I can. The other is that I know the problematics involved in making these effects. My company created a working prototype 3D graphics chip for now-dead Tritech some five years ago. The evaluation card that had the chip had both TV inputs and an output. In addition to defining the chip's interlace flicker filters, I made some software to the card that did exactly what I described earlier: it scanned an NTSC, SECAM or PAL signal, examined movement and used the detected movement to sync to the actual frame rate in the source material. When executed properly, syncing worked very well and I could create very high-quality pictures for video projectors and computer monitors, as well as very nice video conversions across video systems. 9. End Summary If given original 4:3 or Letterboxed Babylon 5 effects video fields in some known digital video format, I could use an ordinary PC and some self-written software to convert the effects to 16:9 with much better quality than what we have now on DVD. JMS, try me :) .