Background
It's needlessly hard to see a movie on the web because there are no widely-accepted standards for how movies should be encoded as data. Currently popular choices become unpopular later and none of them are well-documented (in a technical sense) or legally in the clear so that all browser programmers can implement them. We end up with a set of incompatible methods to do roughly the same thing.
Ogg Theora+Vorbis can change that but Apple and Nokia are fighting it. They want their preferred patent-encumbered formats to become the standard means for distributing video online. This is a real problem for anyone who can't pay for the relevant patent licenses, which means all free software hackers.
"Ogg" is a wrapper that ties together "Theora" encoded video and "Vorbis" encoded sound. Together, Ogg Theora+Vorbis give users a way to see movies on your computer. Ogg Vorbis+Theora are not known to be encumbered by any patents (the only applicable patent on Theora's predecessor, called "VP3", was licensed for everyone to use in any way they want). Ogg Theora+Vorbis are implementable on nearly all modern computers. There is free software (zero-cost and freely to sharable and modifiable) to make and play Ogg Vorbis+Theora movies. Ogg Vorbis+Theora are a great basis for interoperability and a fine choice to recommend in any standard that uses multimedia files precisely because everyone can use Ogg Theora+Vorbis.
Ogg Theora+Vorbis was in the drafts for the next generation of HTML (the lingua franca of webpages) as a suggestion; nobody would have been required to implement support for Ogg Theora+Vorbis. But that didn't stop the proprietors from complaining.
What's the holdup?
Ian Hickson, speaking for the HTML5 working group, recently announced that he removed the suggested Ogg Theora+Vorbis language from the HTML5 specification. The replacement language says that there should be unencumbered common audio and video formats in HTML but it doesn't suggest any specific means of accomplishing that.
How did this situation come to pass? What happened between the time this spec was drafted and now? Corporate influence, in a nutshell. Apple and Nokia's complaints have found an ear with those working on HTML specs.
What can you do to stop this? Raise the issue far and wide with everyone, even non-technical computer users. It's time that these issues no longer live exclusively in the realm where only geeks tread.
Apple and Nokia never liked the language supporting Ogg Vorbis+Theora codecs. Nokia lied by claiming Ogg Vorbis and Theora were "proprietary". What Nokia probably meant is that the evolution of the Vorbis and Theora codecs were not under the control of a corporate board which Nokia could become a member (and thus have a hand in controlling how Theora and Vorbis improve). Ogg Theora+Vorbis specifications are available for anyone to use for any purpose.
Apple's representative in the discussion, Maciej Stachowiak, expressed concern that Ogg Vorbis+Theora might be encumbered by patents we don't yet know about (commonly referred to as "submarine patents"). This is a bogus argument for a few reasons:
-
No modern software is 100% verifiably free from patent infringement.
If one fears submarine patents, one should not turn on a computer (users are at risk for losing a patent infringement case too). Raising fear that a patent holder might sue is a means of trying to stop the free software community from participating as equals in the making of relevant standards. In cases where known patents are actively defended (like the Fraunhofer patents covering MP3), there's good reason to be cautious. Submarine patents are closer to the opposite of that case.
- The MPEG licensing authority (MPEG-LA) doesn't guarantee that their patents cover everything
Hub put it well in a response to a blog on this topic:
Concerning submarine patents over Ogg and associated codecs, this argument is total bullshit. Even the MPEG-LA does not offer any warranty that they are licensing all the patents necessary to implement the specification. In short, even if you pay the patent protection money to the MPEG-LA gangster, there is not insurance that another gangster won’t be coming after you for another protection racket.
Just how at risk are the companies that paid off the MPEG-LA gangster? Xiph, the organization that made Ogg Vorbis and Theora, pointed out some recent MPEG-related patent threats:
Despite the MPEG proponents' claims that MPEG-licensed codecs protect against liability, patent disputes involving MPEG codecs have occurred as recently as the past few months. For example, Lucent v. Gateway and Multimedia Patent Trust v. Microsoft Corp.. The MPEG-LA's own sublicense disclaimer warns that licensees are not protected from patent-related litigation nor are they protected from submarine patents.
-
We don't need what is technically "the best".
Apple contends:
Even if all browsers end up supporting Ogg Theora/Vorbis, these are not the best-compression codecs available. So a large-scale video content provider that wants to save on bandwidth may wish to provide H.264/AAC content to those browsers that can handle it, even if all browsers could handle a lower-quality codec as well.
H.264/AAC are patent-encumbered formats Apple has licensed and uses frequently. No matter how "best" is defined, what people need is something common, unencumbered, freely-implementable, and good enough most of the time. Ogg Vorbis+Theora qualifies and Apple's preferred codecs don't. Ogg Vorbis+Theora is quite good at what it does, plenty good enough for watching clips or entire feature-length movies on a computer. YouTube became famous with far lower-quality audio and video. Software is continually upgraded; we all accept that we are frequently running not-the-best most of the time.
-
If you build it, they will come...
Apple contends that
Many mobile devices cannot practically implement decoding in software, and rely on custom hardware which can handle only a fixed set of codecs. While hardware decoders for H.264 are widely available, I don't think there are any for Ogg Theora.
Even if that's true, it's no reason for discriminating against a free software codec like Ogg Theora+Vorbis. Increased use will spur people to make more hardware to record and play Ogg Theora+Vorbis videos. There was a time where H.264 decoding wasn't popular, I guess we're supposed to consider ourselves lucky that didn't stand in the way of anyone implementing such hardware?
What's important is the critical difference between a proprietor's interest (restricting the number of competitors to those in a similar position when outright elimination isn't possible) and software freedom (letting everyone run, share, and modify computer software, and in doing so socially organize for our mutual benefit). Proprietors would prefer that everyone switch to using what that proprietor can deliver today: for Apple this means QuickTime and QuickTime-compatible codecs, Nokia licenses various codecs and makes money from selling patent licenses to their own codecs. But since no proprietor can swing total dominance, each will settle for keeping the status quo of nothing-works-universally for as long as it benefits them because each proprietor recognizes that that confusion keeps their software viable for their portion of the multimedia audience.
A Prediction
Once the patents covering MPEG development have expired worldwide, Apple, Nokia, Microsoft, and other corporations will lose interest in these formats because the exclusivity has vanished. Despite their distractive arguments their interest here has nothing to do with perceived quality of video or audio, hardware availability, or irrational fear of submarine patents. A proprietor's main interest has primarily to do with sustaining monopoly and keeping competitors away. These companies want to make sure that the public is in no position to mount an effective threat to the established multimedia codec powers that be. Microsoft, Apple, Nokia, etc. don't like dealing with each other as it is, but they'll be damned if they're going to let people like you compete with them too.
fsdaily.com | 30-Dec-07 at 3:34 am | Permalink
Story added…
Your story has been submitted to fsdaily.com! Come and promote your article by voting for it here on FSDaily! Let your readers know they can vote for your story too….
Geoffrey Sneddon | 30-Dec-07 at 4:47 pm | Permalink
Can you find one place where Apple have said they want a patent-encumbered format (note that Theora is patent-encumbered — it just has a RF disclaimer on the On2 patents)? They have said multiple times on the WHATWG and HTML WG lists that they think that it would be unacceptable for HTML5 to require encumbered technology without an appropriate grant of license.
This is totally inaccurate. Ogg is a container format that can contain any video or audio codecs (including DRM-encumbered ones). The three are unrelated apart from the fact that Theora and Vorbis are normally put within an Ogg container, and that they are the two most common codecs in the container.
Again, Theora is known to have patents held by On2, which have a RF grant on them.
It’s pointless if people don’t implement it — it means as much if everyone ignores it as if it isn’t there. The aim is interoperability — this requires everything to support something. Ogg/Theora/Vorbis (from hereon Ogg/etc.) don’t achieve that, as the biggest and third biggest UAs are unwilling to implement it.
Corporate influence only insofar the fact that it doesn’t create interoperability (which must be the aim of any spec, otherwise the spec is meaningless).
Nobody was arguing that. What Apple, Nokia, et al. are arguing is that there is no technical advantage to them, and there is no market incentive (as there is very little deployed content that uses Ogg/etc.) to increase the already large risk they carry by implementing what they already do. Video codecs are known for their high risk — the normal reason for implementing a new one is its technical superiority.
I haven’t heard anybody say that submarine patents are a risk for MPEG codecs (just look at Alcatel-Lucent v. Microsoft). What the major companies have already done is take that risk — some patents are so widespread that any submarine patent would have most likely surfaced by now — and adding any new codecs, as I’ve already said, is a large risk.
If you read what you’re quoting it’s merely stating the H.264 will likely be used, even if there is a fallback for the browsers that don’t support it; something along the lines of
<video src="h264.mpg"><video src="baseline.codec"></video></video>.Apple has already shipped over one million iPhones: they can’t add a hardware decoder to them. This means that an HTML 5 with a MUST support Ogg/etc. (it will be a MUST whatever codec is chosen) can never be supported on a million devices. This is no minor thing to ignore. Even a second-generation iPhone would take time to supplant the current iPhone’s marketshare.
To take those companies in order:
Apple: I’m sure Apple would love more companies to support MPEG standards: what do they have to lose? What they have to gain is obvious: more content out there for their media players (i.e., iPod/iPhone), so more sales. The number of devices (of mobile media players) won’t increase massively: the groups that have the money to develop such devices already do, and have enough money for MPEG-LA’s fees; this mean there is little more competition, and I’m sure Apple is more than happy with the status quo of the media player market. Apple will, however, likely continue implementing technically better media standards as they are developed.
Nokia: Again, more content for their devices. Same as above.
Microsoft: MS doesn’t even support MPEG 4 directly in Windows — the only MPEG codec included with Windows is parts of MPEG 1 (most prominently MP3). MS continues to push it’s proprietary WMA/WMV. They have little interest in MPEG.
Ian Hickson | 30-Dec-07 at 6:56 pm | Permalink
As you say, both Ogg’s suite of codecs and MPEG’s suite of codecs are equally likely to be the target of submarine patents. The difference for companies like Apple is that they have already taken the risk with MPEG — if there are submarine patents, they are going to get sued regardless of what else they do with MPEG. But with Ogg, they haven’t yet taken the risk, so supporting Ogg would mean an _additional_ risk.
BTW, you mention “corporate influence” as if it’s a bad thing. It’s not. The WHATWG works closely with all the browser vendors who are willing to work with us, whether corporate or not. As it happens, there are some companies (Opera, Mozilla) who want Ogg, and others (Apple, Nokia) who find Ogg presents an unacceptable additional risk.
We want a spec that _everyone_ can implement. If one of the browser vendors, big or small, says that they simply will not implement a particular part of the spec, then that part of the spec has to change.
I hope that helps clarify the situation. :-) Please do feel free to take a more direct part in the work, you can find instructions on joining the effort here:
http://blog.whatwg.org/w3c-restarts-html-effort
Or you can e-mail me your feedback on the spec directly at: ian@hixie.ch
Boycott Novell » Time for Ogg (BoycottNovell and the WeBlob) | 30-Dec-07 at 10:55 pm | Permalink
[...] is another good summary of recent events. Such events sabotaged attempts to help Web standards evolve to facilitate openness. Ian Hickson, [...]
J.B. Nicholson-Owens | 01-Jan-08 at 1:52 pm | Permalink
Sneddon & Hickson: First, thanks for taking the time to follow up.
I think the most important point to take away from this episode is the extent of corporate influence in spec drafting and that the deleted language recommending Ogg Theora+Vorbis obviates all of the fears about patent risk. Since Ogg Theora+Vorbis was never required, any patent-based fears (whether rational or irrational) fall away because if you don’t want to implement Ogg Theora+Vorbis you don’t have to and you can still comply with the required parts of the spec. Even if the Ogg Vorbis+Theora language was listed as required, implementers could choose not to implement that part of the spec. There’s no way for the HTML5 spec to compel implementers to do something. But spec authors have the bully pulpit and can use it to recommend things that are in the public interest. Using Ogg Vorbis+Theora is one such idea.
I think where both of you and I disagree the most is that I prioritize business desires lower than our needs as computer users and members of a cooperative society. I don’t see it as my job to look after businesses and I don’t agree that society shouldn’t support bland or vague specs in deference to their monopolistic desires. I find that a recommendation of support for a free codec in a spec is so little to ask on that line.
Now on to the minor points: The fear of increasing submarine patent risk isn’t convincing. If we are to believe that these companies fear increasing submarine patent risk and if we understand that such risk comes from implementing anything, we must also believe that these companies will never implement anything new. Even a new format (for anything) which they devise places them at risk of losing a patent infringement lawsuit. But we know that they will not stop development of their software; they will add new features which implement reading and writing files in new formats. So we know that they don’t mind taking on submarine patent risk for some things which the public can fully implement (ODF and PNG come to mind; Apple’s proprietary spreadsheet can now read ODF spreadsheets and Apple’s QuickTime software can read PNGs). Hence we have to look elsewhere to understand their objections to Ogg Theora+Vorbis.
Ian Hickson: If any part of the spec will be altered to suit any programmer or organization that says they cannot implement something, you’ll end up with a very weak spec that doesn’t address the interoperability users need. History suggests that proprietors won’t be willing to negotiate from a lowest-common-denominator perspective when deciding what to put into their software. Software patents have the effect of helping the patent holder avoid the ravages of a more free market. I wonder if the W3C has really come to terms with the dilemma facing them regarding software patents.
Geoffrey Sneddon: Much of your response is nitpicking, some of it doesn’t acknowledge what I said about the VP3 patents early in the post, and perhaps you didn’t glean that I’m attempting to speak to a wide audience here (including non-technical people). In the interests of brevity and sticking to my point, I simplify the technical parts to what’s relevant for the discussion at hand. That hardly constitutes being “totally inaccurate”.
Future generations of hardware can come with support for things current generations lack, this is no reason to reject what’s in the best interest of the public. My XO-1 computer (the One Laptop Per Child machine) makes and plays Ogg Theora+Vorbis movies and that makes me wonder what other low-power computers can do without additional hardware. However the primary issue here is still the effect on the public and user’s freedom to completely control their devices.
Finally, more MPEG licensees does not necessarily include the public. MP3 and MPEG-LA’s licenses are all structured to exclude free software hackers. The point I should have made in the last paragraph should not have focused on MPEG licensing so much as patent encumbrance that stands in the public’s way of implementing free software because my point hinges on exclusivity and restricting competition, not a particular format or codec. Making movies to play on a device doesn’t necessarily include treating users as equals in development.
Ian Hickson | 03-Jan-08 at 9:59 pm | Permalink
The need we have is for a common codec. Anyone is free to implement any codecs they want, the only thing we care about in the spec is the common codec. Ogg didn’t fit the bill because some vendors didn’t want to implement it — for whatever reason, it doesn’t actually matter — and so we couldn’t require Ogg. The text was in the spec for a while to, as you say, try to bully Apple into implementing Ogg anyway. But they didn’t cave, they said they would not do Ogg and that was that. So I removed the text.
This doesn’t stop Mozilla, Opera, etc, implementing Ogg if they want to.
I don’t prioritise company desires above user needs — quite the contrary, I don’t care about company desires at all, and this is exclusively about the need for users for their to be a single common codec.
Re submarine patent risks: As you say, implementing anything introduces new risk. However, the risk may be worth it (as when it comes to a new feature), or it may be minimal (as is the case with most features). For Apple and Nokia, with Ogg, the risk is not worth it in their eyes — there are other codecs that they have already implemented to address the needs of video — and the risks are high — codecs are a minefield of patents.
For what it’s worth, I have good reason to believe that patent risks really are the concern here, at least on Apple’s behalf, but in practice it doesn’t matter if you believe that or not. It doesn’t change that Apple said “no” to implementing Theora, and that we are thus left with Ogg not being a suitable common codec suite. Apple are actively looking for a codec that _is_ suitable to everyone, so I don’t see this as an act of bad faith.
Finally, you say that the HTML5 spec will be very weak if it takes into account the limitations of all the browser vendors. I do not think this is true. I think in fact it will make the spec stronger, and I think that the broad desire from at least three browser vendors to implement parts of the spec already, years before it is finished, speaks as a testament to the spec’s strength.
There are many specs that didn’t take into account what the implementors wanted. They end up irrelevant and ignored. I have no desire to let my work end up irrelevant.
(PS: please e-mail me if you reply to this so that I can see your reply: ian@hixie.ch)
J.B. Nicholson-Owens | 05-Jan-08 at 12:41 am | Permalink
And so by adopting a line of letting a “no” trump what’s beneficial to the public, you’ll never get in the way of any proprietor. That policy is how you prioritize the patent holder’s desires above user’s needs.
I never said anything about bullying any implementer into anything; I think you misunderstand the phrase “bully pulpit”. A bully pulpit means “a terrific platform from which persuasively to advocate an agenda [...] The term has no relationship to the word bully in the sense of a “harasser”.”.
The rest I have already pointed out before but you seem to repeat without acknowledgment: the language you removed from the HTML5 spec was a recommendation not a requirement. A lot follows from this point despite you claiming the opposite.
Apparently you believe Ogg Theora+Vorbis was required in the language you removed. When something is suggested and not required in the spec it’s fine to let any implementers decide they don’t want to implement the recommendation (no matter what the reason) and leave the recommendation in the spec. Hence all of the patent debates fall away. Let anyone say whatever they will about why they can’t implement Ogg Theora+Vorbis, and then let them comply with the rest of the spec’s required portions and claim full compliance with the required portions of the spec. That would be fair.
Entertaining a distraction, one might look at the patent arguments to see if they hold water. One quickly sees another problem: If you genuinely believe the rationale you’ve explained here about why any implementer wouldn’t choose to implement Ogg Theora+Vorbis, it’s not clear why there would ever be a common video codec you’d find comfortable suggesting in a future spec. Apple has no incentive to find such a codec, they can afford to license anything they want to include in their software. Theora and Dirac are the only viable free video codecs right now and Dirac isn’t ready for widespread use right now. I have no evidence to show that Apple had anything to do with developing either Theora or Dirac. Any implementer who wants their codec(s) to become popular (and thus get lots of licensing fees) can simply take out lame patent-based excuses to help explain why they’re going to work to preserve a chance at a monopoly windfall.
Finally, it does matter why they won’t implement something in the public interest (as FLOSS is)—it points to how some implementers are not behaving in the best interests of users. To remove a recommendation and then continue to talk about it (and make points from the perspective) as if it were a requirement (”and so we couldn’t require Ogg”), your priorities are clear and they don’t favor users seeking freedom. Whether implementers are free to use Ogg Theora+Vorbis was never an issue; they all knew they were long before you said so. The question is whether those writing specs have the guts to do what’s in the best interest of the public.