Feel the source

A good majority of the FOSS crowd seems to argue that distributing source code is always a great thing. Well I'm here to say that it's not all peaches and cream. The OSS convention of primarily distributing source code packages has many downsides that nobody seems to want to talk about.

First and foremost, the OSS model forces downstream projects to make decisions that should be made by upstream developers. What are those decisions? for example, which compiler should be used? what compiler flags should be used? what versions of libraries should something be built against? When compiling software for distribution, there are many such secondary decisions that need to be made. It makes a whole lot of sense for these decisions to be made by those who wrote the code, i.e. the upstream developers. But in the innovative Linux world, most of the decisions are made downstream, at the distro level, by people who don't really understand the code but know how to make an rpm. Even if they are somewhat familiar with the code, surely, the original authors of the code know more. Yet these package maintainers are the ones making these crucial decisions that can affect the performance and stability of the final product. This isn't freedom of choice. This is giving other people the freedom to fuck things up.

Number two. Source distribution dilutes upstream testing efforts. Any software tester knows that as far as testing and QA is concerned, different binaries are effectively different test cases. Each time a downstream project patches a program's code, or compiles it in a different way, they're essentially creating a fork of the code. This instantly undermines all of the testing that was done by the upstream project. Even fossclowns can see that this can cause problems. In the most obvious cases, you end up with something like the Debian OpenSSL debacle. In the more subtle cases, you get end users reporting problems because the distribution used a compilation flag that the upstream developers didn't test, or because they used a different library version than what was on the developer's box when he did a svn commit. They said FOSS would be better because it gets globally distributed testing of shared code. In actuality, everyone is testing slightly different binary translations of the shared code, so any benefit is effectively spread across these many variations.

A simultaneous effect: source code distribution also causes duplication of effort downstream. We've already seen that every distro has to do everything themselves in their own way, and that surely includes to solving all the same problems that every other distro faces. Which compiler should they use? which library versions should they use? Which versions of the pieces work well together? Not only are the hundreds of distros not sharing their testing and integration efforts, they are actually duplicating those efforts many times over. So much work that can't be re-used by other distros, because everyone's configurations are different. It's a huge waste. Guess why even as the number Linux users and testers increase, the quality of distros doesn't really increase proportionally? Every new distro just adds to the waste and does all the same work as everyone else. But hey, all the work is free right?

Things aren't all rosy from the developer's point of view as well. But how can that be? I mean, developers love source code, right? Well I'm glad you asked.

One common thing you'll see with OSS libraries is that they don't come with very good documentation. It's OK though, the dev's say, since you can just read the source code. To someone who doesn't write code, that may sound just fine. Who needs docs when the code says exactly what it does right?

Wrong, unfortunately.

Let's put aside the problem that reading source code to figure out what something does already totally sucks. No, I'm saying that as far as documentation for API's go, the source code is often too specific. When a developer can see the implementation of a particular interface, he can also very easily and inadvertently start to depend on the particular characteristics of the implementation (often characteristics that the implementor didn't even intend in the first place). Source code "over-specifies" an API. The worst is when developers start depending on the bugs.*

Of course, the proper way to do this is to document your libraries and interfaces and hide the implementations. Make your assumptions clear, yada, yada. There are tons of software engineers that understand this. The problem in OSS is that having the source code serves as an excuse not to create this documentation. "Just look at the code" they say, as if its the same thing. That's where both library consumers and implementors get into trouble.

Another more mundane problem. How do developers recoup their development costs while giving all their "secret sauce" away? This issue has been debated to death in other forums, but it's still worth mentioning here. Yes, I'm tired of y'all telling me that Google made cash-money off of open source. But you know what? they're not selling software. If your goal is to produce pure software products for end users to consume, what's the business model? I'm sure someone's gonna bring up Redhat, but do you want me to go over the Redhat vs MS vs adobe revenue numbers again? In the end, while there have been some entrepreneurial successes around open source, there have been very few, if any, where a company wants to sell software itself. (And getting bought by Sun doesn't count)

Last point. The fact that open source projects mainly distribute source code actually prolongs all the problems above. If all OSS projects were to actually try to distribute binaries, they'd realize how fucked up the situation actually is (and maybe they'd even try to fix it). And if OSS devs can show that they can coordinate enough to actually distribute their own stuff in binary form, then surely commercial companies will soon learn to play along and join the fray, benefiting everyone. But I'm definitely not holding my breath.

Note that I'm not saying that source distribution doesn't have it's place. It's useful for fixing bugs, and coordinating improvements to upstream code. But too many fosstards dismiss binary distribution without acknowledging its inherent benefits. That's just ignorant.


* Note that, not having source code doesn't entirely prevent this. There are many famous examples in Win32-land where programs depend on bugs within the runtime, I claim that similar mistakes are strictly and significantly easier when source code is available.