Open Source Video
The Web Video Turf Wars
By Trevor Marshall
September 11, 2000
Last week, I put together some video clips for my daughter's Web page.
I used the Real Video encoder to create the RealVideo content, but because the quality was so poor, I also encoded the clips with the FreeWare Tsunami MPEG encoder I described in last month's column.
Before sending the MPEG clips to the Web server, I previewed them with my Windows Media Player. They were quite large, too fast to stream, even with my cable modem, but the media player did quite a good job of buffering the download. Since I had used MPEG1 for the encoding, I figured that practically everybody would be able to play them (on Mac, PC, or Linux).
Imagine my disgust when I clicked on the Web page link. It was the Real Networks player that took control of my MPEG stream, and promptly commenced to tell me that .mpg video was an "unsupported format" it did not recognize.
After several hours of trial and error, uninstalling and reinstalling the various media players, I found the filetype .m1v would not be snatched by the Real Player, and would display, as intended, using Windows Media.
But why should I have to do this? Why would Real be so keen to grab .mpg files when it is only capable of playing a small percentage of them? Why didn't it leave the .mpg file type to a player that can handle it properly? I spent several hours cursing the fact that I had accepted Real's "free online upgrade," which my player had been nagging me about for many months. I should have held out. My old Real G2 (beta) player used to only lay claim to the media types it could actually play!
Take No Prisoners, Win At All costs
Steve Gibson (of SpinRite fame)
has reported that his exact personal name and e-mail address was being sent to Real Networks whenever he downloaded a file using RealDownload. Additionally, Real was getting the complete URL address of the file along with a unique ID tag that had been assigned to his machine when he installed Real's player software.
How did it get Steve's name and e-mail address? Real told him that when he had purchased a Real product (some time ago) a "cookie" was transmitted back to his machine (unencoded) carrying the name and e-mail sections of his secure purchase transaction.
At this point, Real Networks officially denies that any such demographic data gathering is going on. Personally, I will bet my life that Steve's version of the events is accurate.
Microsoft wants its own "Advanced Streaming Format" to become the dominant method for transmission of video over the Internet.
Following my mention in last month's column of Avery Lee's VirtualDub video utility I received a letter from Douglas Bostrom "I was struck by your references to VirtualDub, which was the subject of a recent attack by Microsoft. MS appears to find it threatening that VirtualDub, and other [Open Source] programs like it, offer the possibility of moving files from their [proprietary] ASF format to other storage methods. See the whole sad story at: http://www.advogato.org/article/101.html"
Avery Lee had reverse-engineered Microsoft's Advanced Streaming Format (.asf) and added it to the native .avi and .mpg support in his excellent program. For a while, VirtualDub had been able to take media in Microsoft's .asf format and convert it to the more widely used .avi and .mpg formats. But Avery received a phone call from Microsoft, threatening to take legal action against him if he did not remove the .asf support from VirtualDub. Avery was smarter than to argue with Microsoft's legal team, and immediately removed the offending code. You can read more of the details in Douglas' link above.
In addition to its .asf patents, Microsoft has also been using its excellent MPEG-4 video codecs to help it dominate streaming video. Microsoft helped develop the MPEG-4 specification, and at an early stage it put out a codec that, at this present moment, is clearly the best video encoder available (in my earlier columns I had put together some video clips to demonstrate this superiority, but I have had to temporarily take them down until I work out why Earthlink is charging me an arm and a leg to host them).
Microsoft decided to keep its Version 2 and Version 3 MPEG-4 encoders proprietary, and use them to promote the growth of the ASF format.
According to Avery Lee,
"Microsoft does not want the MPEG-4 V3 format used in .avi files, only in .asf files. As a result, attempting to use the MPEG-4 V3 compressor in any AVI program will give you a -100 (ICERR_ERROR) when you try to start the compression process. If you attempt to bypass this lock and compress to MP43 anyway, the DirectShow MPEG-4 decompressor used by Windows Media Player won't play it."
But Microsoft had released early versions of this MPEG-4 codec to developers in a beta distribution of their Media Toolkit.
These early MPEG-4 encoders were NOT locked into only using the .asf format. As Avery explains "Technically, if you still have the Windows Media Tools 4.0 beta distribution, you have a version of the MPEG-4 V3 codec that can still encode to AVI, and is legal to use. Unfortunately, you still can't distribute the codec outside of the WMTools distribution, and all the newer Windows Media tools will upgrade the codec, thus making your files unplayable once again. As far as I know, the only legal way to encode to MS MPEG-4 V3 is to use the .asf format"
And it goes without saying that the V3 codec produces significantly better video than the earlier V1 and V2 codecs.
The Internet Routes Around The Obstruction, And DivX Is Born
I am reminded of John Perry Barlow's classic statement "The Internet treats censorship as damage and routes around it," as I contemplate how these Beta Microsoft codecs disappeared into a network of underground developers and re-emerged as the "DivX" format that has been setting the standard for video distribution across all computing platforms. Versions of Gej's original "DivX" MPEG-4 encoders are now available for Linux, for MacOS, for beOS and for Windows.
Microsoft's MPEG-4 V3 encoder was "re-engineered" as the "DivX Low-Motion" codec, and the V2 as "DivX Fast-Motion." The Linux DivX player uses the same DLL object code as the PC version, but clearly the port for the Macintosh demonstrates that these underground developers have access to source code. In fact, several MPEG-4 source codes are linked from a new website, OpenCodex.com, that is offering a $50,000 prize to the first developer who produces a good MPEG-4 codec for Quicktime, and releases it to the public domain (it's nice to see that Quicktime devotees will not be bypassed by the new technologies).
Another commercial venture, "Project Mayo" has taken in "Gej," the anonymous originator of the DivX concept, and is funding development of new high-performance video codecs that will be made available, across all platforms, in an open source distribution.
But if you were to click on a DivX encoded data stream and Windows Media Player popped up to display the video, it complains that it has no available codec to handle the "div3" data stream.
You see, each codec has its own individual signature, or FOURCC code, embedded into the header information. Windows Media Player goes out onto the Internet, looks up the "div3" FOURCC code on a master list at Microsoft's offices, and finding that DivX is not an approved codec, it will refuse to play it. You have to manually install a copy of the codec on your computer to be able to view DivX content.
Will Windows Media Player also refuse to play video from the newopen source codecs? This is the battleground on which Microsoft intends Windows Media Player 7 to win. Microsoft wants its player to be so comprehensive that you will be happy using it to display all your media files. And if you are unlucky enough to click on a link to a RealVideo file or an open source file, you can bet your bottom dollar that an error message will materialize from the FOURCC lookup process! This is the battleground that caused my new RealPlayer to try and grab the MPEG files.
Unfortunately, I think that the battle for the player will get nastier before it gets resolved. If CBS or NBC controlled virtually ALL the television you were watching, just how much could they charge for advertising? What a windfall it would be for Microsoft or Real Networks to control Internet media creation and distribution. And what a disaster for technical innovation.
Open Source vs. Microsoft And Real
Clearly, there is a lot at stake in this battle. It is not just that the anti-Microsoft forces have united around one ethos in a manner reminiscent of the development of Linux itself, it is a fundamental demonstration of the importance the Internet community places on the development of full-screen video transmission. Some of those developers, such as Gej and "Capt Stux" have even been prepared to work underground, without adulation, focused on the goal of creating open source video encoders and decoders that are unfettered by the licensing conditions that Microsoft, or anybody else, might try to place upon them. It is hard to see how anybody can stop them. And digital video will never be the same again.
Although he has been hacking for longer than he likes to remember, Trevor Marshall's first articles for Byte didn't start until 1985. Prior to that, he had put together the Thousand Oaks Technical BBS using a Z80, some huge 5-meg hard disks, and a modem made from several pieces of wet string (that string was left out to dry in the late 1980s, after the birth of ARPAnet). Other trophies include a few years moderating on BIX, a 1966 amateur-radio record for long distance communications on the 576-MHz band, stints doing sound amplification for Bon Scott, and a vanity e-mail address on The Well. Trevor retired in 1988, when he founded YARC Systems, a company that has retained him as its chairman and CEO to this very day.
For more of his columns, visit theTrevor's View Index Page
