Toward a New Compendium of Knowledge (longer version)

1. Thinkers of the world, start imagining.

According to one source, there are over one billion (a thousand million) people on the Internet.  That means there must be tens of millions of intellectuals online--I mean educated, thinking people who read about science or ideas regularly.  Tens of millions of intellectuals can work together, if they so choose. Technologically, there is nothing standing in their way.  This is truly a dumbfounding, incredible situation.  Humanity has not even begun to explore the possibilities inherent in this situation.  For all the endless talk about "Web 2.0," we have hardly even begun to think about harnessing this intellectual power for knowledge-building projects, much less actually act on it.

Imagine what is possible with tens of millions of intellectuals working together on educational and reference projects.  (The trouble, of course, is actually getting them together--"herding cats" is the operative phrase.)  What could attract them to pool their intellectual power?  What content creation systems would best harness this power?  What fantastic things might result?  Imagine the discussions, the journalism, the news summaries, the textbooks and educational material, the encyclopedias, the libraries, the multimedia, the 3D universes, not to mention brand new sorts of resources possible for the first time and only through massive collaboration.  The paltry first steps we have made on such resources are admirable, but ultimately will prove to be laughable compared to what will exist in five or ten years.  It will resemble the difference between the first PCs and the firepower of the latest business-class desktops.

Whenever I think about this now, I literally quiver with excitement, and I am amazed that we, educated people throughout the world, have barely begun to imagine what new reference and educational materials could come into being, if we pool our efforts in the open, collaborative ways demonstrated by open source software hackers.  Even less have we begun to take such possibilities really seriously, or actually get to work on them.

But this is changing very rapidly, and I want to make a prediction.  In the next year, by the end of 2007, every major university, library, museum, archive, professional organization, government, and corporation will be asking themselves with increasing urgency: how, using what systems and methods, can we pool the entire world's intellectual resources to create the ideal information resource?  What worldwide projects and organizations should we join or help to create?

2. Wikipedia.

We may take Wikipedia as an early prototype of the application of open source hacker principles to content rather than code.  I want to argue that it is just that, an early prototype, rather than a mature model of how such principles should be applied to reference, scholarly, and educational content.

Wikipedia, started only five years ago, now has millions of articles in over one hundred languages, and has nearly singlehandedly introduced the world's intellectuals to the possibilities of enormous collaborative efforts.  It is a project that shouldn't work, but does--who could have expected such a radically open project to produce anything of value?  But, by giving intellectuals the world over an open platform on which to work together, a clear task, and a simple interface, Wikipedia has shown a global audience what enormous, distributed knowledge collaboration can achieve.  The work of the Wikipedians has astounded the world.

I always have been an enormous fan of Wikipedia, and I still am.  But I have achieved notoriety with some of Wikipedia's community and supporters by declaring that we can and should do better.  I now want to help launch something better, if that's possible.  Let me explain why I am doing so, and then I will describe the project itself.

When on January 2, 2001 I first had the idea for Wikipedia, and then got to work defining policies and leading the community that built it, I was also employed as editor-in-chief of Nupedia, a more straight-laced, peer-reviewed free encyclopedia project.  Wikipedia was to be a free-wheeling, fun side project, a dynamic source of content, for the more serious Nupedia project.  Wikipedia was closer in spirit to the old open source, hacker ways, where Nupedia was essentially an academic project.  But together, Nupedia and Wikipedia were going to be an "unstoppable high-quality article-creation juggernaut," or so I said in 2001.

I will not go into the history very much, which I've done elsewhere, but I want to recount a few relevant details.  Contrary to popular belief, Nupedia did not fail simply because its system was too complicated and closed.  Rather, in 2001 and the beginning of 2002, I was working so hard on Wikipedia that I was greatly overextended--and so, Nupedia effectively lost its editor-in-chief.  Few people know, or remember that: Nupedia withered in large part precisely because I was focusing on Wikipedia instead.  But by the end of 2001, the Nupedia Advisory Board had agreed to relaunch the project with a greatly simplified two-step system.  That's another thing a lot of people don't know about Nupedia.  However, by that time, the Internet economy had collapsed and so Bomis, the company of which Jimmy Wales was CEO, and which hired me to start these encyclopedia projects, lost the ability to pay me.  I had to resign.  This hit Nupedia much harder than Wikipedia, since Nupedia was much more of a top-down project.

After that I twice offered Jimmy Wales a way to keep Nupedia alive.  Once, I offered to find an academic sponsor--a university or foundation--for Nupedia.  Another time, I offered to find the money to buy the domain name and subscriber list myself.  Jimmy essentially turned down both offers.  I could have saved it, and I wanted to save it, but Jimmy did not support the idea.  It seems that with Wikipedia, he had found the only model that he wanted to survive.  But the result was that only half of the original conception of "the finest encyclopedia in the history of humankind"--the wild-and-woolly half--was preserved.

Not surprisingly, with only one-half of the original design, some problems with Wikipedia--problems that continue to this day--emerged.  Wikipedia quickly showed itself to have a wonderful system for producing massive amounts of reasonably good content quickly.  But that does not mean that, as an encyclopedia and as a community, it is free of serious and endemic problems:

  • The community does not enforce its own rules effectively or consistently.  Consequently, administrators and ordinary participants alike are able essentially to act abusively with impunity, which begets a never-ending cycle of abuse.
  • Widespread anonymity leads to a distinguishable problem, namely, the attractiveness of the project to people who merely want to cause trouble, or who want to undermine the project, or who want to change it into something that it is avowedly not--in other words, the troll problem.
  • Many now complain that the leaders of the community have become insular: it has become increasingly difficult for people who are not already part of the community to get fully on board, regardless of their ability or qualifications.
  • This arguably dysfunctional community is extremely off-putting to some of the most potentially valuable contributors, namely, academics.  Furthermore, there is no special place for academics, so that they can contribute in a way they feel comfortable with.  As a result, it seems likely that the project will never escape its amateurism.  Indeed, one might say that Wikipedia is committed to amateurism.  In an encyclopedia, there's something wrong with that.

Can Wikipedia recover from these problems?  The first step to recovery is admitting that you have a problem in the first place.  Wikipedia's most passionate defenders, if they react at all, will probably do nothing but explain why I am mistaken in each of these criticisms.  There are some active Wikipedians who are able admit at least some of these criticisms.  But could the Wikipedia community as a whole admit any of them, with enough force actually to do anything about them?  I am now, I fully admit, an outsider looking in on the project.  But I do still watch the project a fair bit, follow the news about it, and read mailing list posts and events on the wiki.  So I know of course that there are movements afoot to reform Wikipedia in various ways.  But I see little evidence that the community, whatever its discontents, will go so far as to admit the problems I've listed.

The failure to recognize these serious problems is a reflection of the fact that, at bottom, they are political problems.  Like all open communities online, Wikipedia's community is self-selecting, and its policies have determined who stays and who leaves (or is driven away).  For this reason, online communities tend to become rather conservative in their attitudes toward their own systems, and Wikipedia is certainly no different.  So it is not surprising that, as anyone who is aware of how Wikipedia works knows, the changes made to the system recently have been mostly cosmetic, and even the bolder of these changes have little chance of solving the problems I outlined earlier.  The first step to solving a problem is to admit that it's a problem; and much of the Wikipedia community will not admit the problems I've listed, unless they have a massive collective change of heart.  And, I think, that is very unlikely ever to happen.  In fact, you could say that I have waited for several years for it to happen, and it never has.

3. A new community and a new project: the Citizendium.

[Note, January 2007: while we are still full steam ahead with a fork of Wikipedia's processes, we are having second thoughts about forking Wikipedia's articles.  We've decided as an experiment actually to delete all the unedited Wikipedia articles from our database, to encourage people to start new articles altogether.  We think this might encourage even more activity than we've had--though we're pleased with the amount of activity we've had so far--and help us to create a more distinctive Citizendium culture.]

Often (not always), if you wish to make any very important changes to an open source or open content project that has an entrenched community, the only way to do it is to start a new community.  And that is what I propose.

I propose a fork of Wikipedia to be called the Citizendium, that is, the Citizens' Compendium.  I doubt I have to remind many people in the audience of this, but the open content license, the GNU Free Documentation License, permits other communities to work on their own versions of the content from the parent project.  So, legally, this project is clearly permitted.  I think it is also morally permitted--perhaps even morally recommended--if there is a chance of retaining Wikipedia's virtues while eliminating the problems I just mentioned.

I propose to start what we might call a "progressive fork." A progressive fork works like this: we will begin with all of Wikipedia's articles, so that the Citizendium will begin as, simply, a mirror of Wikipedia.  Then people start making changes to articles in the Citizendium. On a very regular basis, we will refresh our database with the latest versions of Wikipedia articles.  If the Citizendium has not changed an article, while Wikipedia has, then we proceed to upload the most recent Wikipedia article.  But if the Citizendium has changed an article, then it is not refreshed.  That's when that particular article is forked.  Of course, I have no doubt that tools will be written immediately that will allow users to compare the differences between the Wikipedia article and the corresponding Citizendium article side-by-side.  In addition, of course, people will be able to start brand new articles on topics Wikipedia has not yet covered.

Obviously, you want to know how the Citizendium editorial system will differ from Wikipedia's system.  There will be three main areas of dissimilarity.  First, the project will invite experts to serve as editors, who will be able to make content decisions in their areas of specialization, but otherwise working shoulder-to-shoulder with ordinary authors.  Second, the project will require that contributors be logged in under their own real names, and work according to a community charter.  Third, the project will halt and actually reverse some of the "feature creep" that has developed in Wikipedia.  Allow me to elaborate these differences.

Before I elaborate, however, let me assure you that I do not think that I necessarily have all of these details right.  Obviously, I am fallible and, like all of us, my understanding is limited.  Clearly, this project needs as much intelligent input as it can get.  So I want to start a debate among people who agree with me about Wikipedia's problems and with at least many of the suggestions here.  Together we can settle upon, and get broad community buy-in for, the best set of policies.

First, here are some details of the editorial system I propose:

  • We will have a new role in the system: that of editor. Others will be called authors. Generally, authors will defer to editors when editors are speaking about their areas of specialization.  When authors get into a dispute, they may work out a compromise, or they may consult an editor.  Editors' decisions will be logged in a new, standard part of each article's discussion page.
  • For the most part, editors will work in the system just as everyone else does, "shoulder-to-shoulder" with ordinary authors.  Editors will not be able to direct work in a top-down fashion, or to "squat" on articles and prevent others from making any changes.  Editors who are not comfortable with this arrangement should not participate.  Those who attempt to make articles their personal bailiwicks, shooing off everyone else, will be ejected from the project, no matter what their qualifications.
  • Editors will be self-selecting, in a certain way.  There will be no editor selection process.  Rather, editors will be invited to come to the website and simply declare themselves to be editors, if they meet certain benchmark requirements--the same straight-up credentials that the offline world relies on.  Editors will be required to state their credentials on their user pages, for everyone to examine.
  • It is not the case that there will be one editor per topic or subject.  There can be as many editors on a subject as arrive: the more the merrier.
  • Editors will not place their names on articles, thereby claiming ownership over them.  They must, however, sign editorial decisions that they articulate on article discussion pages.
  • Disputes among editors, and long-standing and difficult content issues, will be settled by discipline-oriented editorial workgroups.
  • Editors will have the right to place articles in an "approved" category.  Other (qualified) editors may remove articles from the "approved" category; again, disputes are to be settled by discipline-oriented editorial workgroups.
  • These editorial workgroups will be staffed only by editors.  They will not be top-down, bureaucratic structures, but flat communities of equals that operate democratically.  There will be Chief Subject Editors, but they will serve the role of facilitators and organizers, not dictators.

Second, some differences in the community:

  • There will be no logged-out editing and no anonymous editing.  Anyone may participate, but all must be logged in under their own real names (we will use the honor principle to begin with), and with a working e-mail address.  Where Wikipedia shares the culture of anonymity found in the broader Internet, the Citizendium will have a culture of real-world, personal responsibility.
  • The community will launch with a charter that articulates the project's goals, fundamental policies, governance, and the rights of participants and of the public.  As with most charters, the Citizendium community charter will be relatively brief and vague, difficult to change, and regarded as "binding law" of the project.
  • All contributors will, as a condition of their participation, be expected to support the community charter.  If they cannot do this, they are not welcome to participate and may be ejected from the project.
  • The charter and rules will be enforced by "constables."  In time, an effective and fair "legal" system will be established.  It will be made up of people who are mature, well-educated, possessed of something of a legal mind, with a record free of major infractions, respected by their peers, and clearly committed to the community charter.
  • Constables will rapidly eject the project's inevitable, tiresome trolls, without going through a long, painful process of the sort Wikipedia suffers under--which it euphemistically calls its "arbitration" process.  A fair and open system of clear rules will allow them to do.  Decisions will, of course, be appealable.
  • Constables will not have any special authority to make editorial decisions, unless they are also editors.
  • Those constables that are also editors will not be permitted to enforce the decisions and recommendations that they make in their capacity as editors.

Third, over the years, Wikipedia has suffered some feature creep.  The Citizendium will be deliberately simplified and kept simple:

  • Subject categories will very probably be eliminated.  Citizendium articles serve as their own category pages, so to speak.  So, when someone edits a Wikipedia-imported article for the first time, he will be instructed to remove all subject category information.
  • The "Wikiproject" method of organizing work will be simplified in the Citizendium.
  • "Portal" pages will probably not exist.
  • Project news will be officially reported in one place: a blog, to which a large number of people will have posting rights.  (This blog has not yet been set up, but will be soon.)
  • So-called user boxes, which have been controversial and much-abused on Wikipedia, will not be permitted.
  • No doubt there will be other simplifications as well.

Fourth, copyright and libel abuses will be handled quite differently.  There will be a zero tolerance policy toward such abuses.  Moreover, the living subjects of Citizendium articles will receive much more courteous treatment than they have sometimes received from the Wikipedia community.  Among other things, this might mean that they would be able to request removal of biographies about themselves--if they are not politicians or other prominent public persons--or even to have a crucial editorial role in the articles about themselves.  Essentially, I want to make the Citizendium a project with which John Seigenthaler, Sr. and Bernard Haisch are comfortable.

Finally, the Citizendium won't officially call itself an encyclopedia.  We might call it an experimental workspace, to start articles and to improve article quality.  It will require a vote of the project's future governing body or bodies for us to call ourselves an "encyclopedia" officially.  Until then, we will be a "compendium."

So much for the differences, and as I hope you can see, they are significant.  But it is important to note that the system will still work very much the way Wikipedia does, in many respects.

Areas of similarity include:

  • It's more or less, kind of like, an encyclopedia.  It's a wiki that aspires to be as good as a real encyclopedia.
  • It's open to virtually everyone.  Virtually anyone can come to the website and, within a few minutes, be working on an article.  It is not Expertpedia.  It's for people who want to work with and under the direction of genuine subject-area experts.  Virtually everyone who is interested in working on an encyclopedia using their real identities, and who agree with the ground rules and aims of the project, will be welcome.  The authors will probably greatly outnumber the editors.
  • People will be able to add significant content while temporarily "breaking"  certain patterns or article flow, and ignore complicated formatting rules, etc.; others can then be expected to do the necessary formatting.  There will still be this sort of informality and division of labor in the Citizendium.
  • It will be free to read and to copy (it will use the same license, the GFDL).
  • There will be no advertisements.  There may be unobtrusive non-profit sponsorship statements.  (See the FAQ for details.)
  • It will come under the control of a non-profit foundation (yet to be either determined or formed).
  • The neutrality policy will be virtually the same, and the rule against original research will be virtually the same.  (These policies go back to Nupedia days.)
  • Many other article policies (e.g., bolding the topic) will be the same.
  • The MediaWiki software will be the same (though some settings will be different) and there are no plans to fork it.

4. The way forward.

[Note, Jan. 2007: this section is now old news entirely and may be safely skipped.]

Finally, I want to discuss the way forward--in other words, how to get the project started.  First, let's talk about the short term, the medium term, and the long term.

I stipulate that, whatever else we do, we should have the servers and the wiki software set up, and available at least to some beta testers, as soon as possible.

While waiting for the wiki to be set up, what can the ordinary rank-and-file future authors or editors do?  Three things: first, join the project forums or a project mailing list (the list called Citizendium-L at least); second, contribute your thoughts to the discussion; and, third, wait for the announcement that the wiki is ready to edit.  I want to start the project with a bang.  I would like to strain the limits of whatever server configuration we put in place.  Also, a donation at this crucial juncture would help get this venture off the ground.

So much for the short term.  As to the medium term, over the coming weeks, I hope the Citizendium project will be contacted by individuals as well as universities, foundations, and companies.  My personal belief, which I do not hold very strongly, is that the latent interest in and support for this project will allow us to collect a truly stellar group of advisors and partner institutions; if so, this experiment might succeed brilliantly.  As to what we need, precisely, please see the project FAQ for some ideas.

Once the wiki has launched, we should begin to form discipline-specific editorial groups.  They will have their own forums or mailing lists and eventually their own meetings.  I want to emphasize at the outset to potential editors that their role in the wiki is not to control the wiki with an iron fist, not to throw up roadblocks to work, but instead to facilitate work by making binding decisions as needed.  There will be rules to this effect.  Editors and editorial workgroups alike must be committed to working within the wiki format--they must not try to change it into something that it is not.

Also once the wiki has launched, I think we should immediately hold a series of monthly face-to-face meetings in about a half-dozen major cities in the United States, Canada, Great Britain, and Australia (or New Zealand)--English-speaking countries, because the plan is to organize the English language project.  I hope the first meeting will be in the San Francisco Bay area (close to where I live) in October, but nothing has been set.

I think perhaps the best way to get these meetings started is if academic departments send the Citizendium project leadership an invitation to host a workshop about the Citizendium. Based on the invitations we receive, we will establish a workgroup to decide when and where these meetings will take place.  At the meetings themselves, then, we will get to know each other and talk about the project.  At the same time, I, and other project leaders who join the project, will hammer out constructive, operational partnerships, do interviews, and so forth.  We will also use the meetings as platforms to promote the project itself to the larger community of intellectuals.

But note that since the project is necessarily widely distributed, and almost no one will be able to attend all meetings, important decisions will not be made at these meetings.  Except for the last meeting.  I conceive of the last of these meetings--perhaps in March, 2007--as a constitutional convention, in which the essential policies of the project and its governance are hammered out and adopted by the advisory committee.

My notion--and at this point, I could be persuaded to do things differently--is to focus first on organizing the English language project formally.  The other Citizendium wikis, in other languages, could be launched at the same time as the English Citizendium wiki.  But my current thinking is that if the management of the non-English Citizendia require my personal assistance, that assistance would probably be best rendered after going through a pilot process in the English language.

But I hope you will interpret (correctly) the fact that I am making this announcement in Germany as indicating my personal commitment to making this project fully international as quickly as is feasible and responsible to do.  On the other hand, I do not want to presume to speak for the interested German-speaking community online.  Knowing as you now do that I personally am getting behind a fork of the English language Wikipedia, I put the question to you: should we also make similar forks of the Wikipedias in other languages?

Anyway, I think that, whatever Citizendia actually are created, each different Citizendium should probably adopt its own charter and have a separate management.  Once the Citizendia in a dozen or so major languages have reached that stage of development, we can have fully international meetings, with delegates from each organization.

By the way, I am not averse to considering the argument that we ought to have these organizational meetings all around the world.  I just want to make sure that the resulting projects have some important, baseline principles in common, and are interoperable.  The whole question of how to achieve internationalization is one I hope we will discuss on one of the project's mailing lists, Citizendium-World.

While this organizational work is going on, I wish to put together the finest possible set of advisors for this project.  Their role as advisors will give them a special platform from which to advise the project and speak on its behalf.  Moreover, it is this group of people who will meet at the constitutional convention to ratify the project's charter.

So, let us create a vibrant yet responsible new community.  Together we will, I think, have an amazing journey.  To the extent that I can influence this new community--which will, I am sure, nearly instantly take on a life of its own--I want to encourage a set of values.  These values may be grouped into sets, as follows:

  • freedom, independence, and neutrality
  • the love of knowledge, critical thinking, and respect for both expertise and for the value and ability of uncredentialed people
  • maturity, personal responsibility, common sense
  • compromise, consensus, and collegiality
  • openness, rejection of insularity, and respect for the rule of law
  • a love of simplicity, a robust dislike for bureaucracy, and not using computer algorithms (or aggregation) where individual judgment is required

The Role of Content Brokers in the Era of Free Content

Draft June 9, 2006; very lightly revised, October 2006

I. The problem of funding content

Ever since entering the open content arena in 2000, I have thought about, and been asked (repeatedly) about, how to pay for free content. This is a pressing problem for professional content creators, because of pressures from the Internet. It is worth rehearsing these pressures and their source, in order to contextualize a proposal I want to make.

Talk, or low-quality talk, is cheap. That's why mediocre content online is so plentiful. But, as with anything, if you put a price on talk, you increase its quality. And that's why there are (and always have been) professional communicators and artists as well as unpaid amateurs: communicators and artists produce work of different levels of quality.

The trouble about the Internet, for professional talkers, is that so many other professional-level talkers are willing to give away their talk for free. Over time, so much high-level talk is available that, as the economics of supply and demand dictate, it becomes harder and harder to pay for high-level talk. So what are the talkers supposed to do for a living?

That's the problem of funding free content. But, as I'll explain next, it's really a problem about funding content, period--of getting money into the hands of content producers, period. After all, oversupply pressures are a very real practical problem for newspapers that are laying off staff, and for reference publishers and others who are made very nervous by the availability of massive amounts of free content available from Wikipedia and other such sources.

II. The traditional publishing brokerage model under threat

I invite you to think of the economics of publishing on the model of a brokerage. A broker is an agent who arranges and settles a deal between a buyer and seller. So think of the publisher as an agent who orders and arranges content from the seller--writer, speaker, artist, or whatever--and takes money from the info consumer, or the buyer.

"Talk brokers" used to be essential to the task of publishing, because most sellers (talkers) could not afford printing presses or distribution networks. The Internet changed that: now anyone with an Internet connection can get virtually unlimited bandwidth on which to rant on, to a potentially global audience, for as long as he wants. But there was still the necessity for editorial services, because the info consumers still needed someone to edit and select credible and noteworthy information. Now that, too, is changing. Aggregation a la Google News, and community editing a la Wikipedia, are providing editorial and selection services for free.

So, while the traditional role of publishers as talk brokers may never completely go away, nearly everyone agrees that it is under an increasing threat. I and many others who think a lot about collaborative content creation believe that it is only a matter of time, moreover, before professionals, including academics, jump with both feet into collaborative content creation. The Citizendium is committed to building an expert-guided version of Wikipedia, and in other ways acting as a clearinghouse for expert-vetted free information. If it succeeds, whither the traditional talk broker role of publishers?

III. A new model of publishing brokerage

Perhaps it's sad to say, but publishers are needed less and less, not because anyone just up and said "We've had enough of them!" but because current technologies and methods have enabled people to get together and perform the same basic functions that publishers have performed. The roles publishers played as talk brokers--namely, their roles as editors, selectors, printers, distributors--are instead played by the general public, from students, to teachers, to civic-minded professional volunteers, and by the processes of collaboration and aggregation. If you are a communicator or artist, free content is (as you probably already know) in your future, like it or not. It might not all be free, but a lot more of it will be than is now the case. This will make it much harder for you, as communicator or artist, to get paid through the traditional content broker model, and harder for publishers to make money through their traditional content brokerage services. We have a new industrial revolution on our hands.

To say this is to describe and explain the problem of funding free content, not to solve it. So, in order to move us toward a solution, I would have us redefine the role of publisher-as-broker. It's not that we no longer need publishers to act as brokers, it's that the nature of the brokerage needs to change.

Let me preface this by saying that I surely can't claim originality, because it's an idea that naturally suggests itself to those who live in this problem space. Besides, there are already examples of this sort of thing in existence (such as Google Answers).

Since first drafting this essay, a promising new effort in this direction came to light: Jay Rosen's NewAssignment.net. The following proposal is a slightly different and more generalized take on the same idea.

So here's the proposal: the public presents an offer for a specific sum to go to someone who will write authoritatively on such-and-such a subject; the broker selects the content creator, who creates the content; and then the broker releases the content to the public, free for all (under, for example, a Creative Commons license). The buyers are still the general public, but are expanded to include groups of people, clubs, schools, universities, organizations, governments, and other entities that pay for the work on behalf of the general public. The sellers are still communicators and artists. The brokers can still include editors, designers, and other publishing industry professionals.

I'll enlarge on how I think the ideal content brokerage system should work, but first, I should explain how this could possibly solve the economic problem posed by plentiful free content. "Suppose there is a system in place," a critic challenges me, "where people can commission works that are then released free into the public domain. Why think that this could economically support the present set of professional content creators?"

My frank answer--the only honest answer, really--is that I have no idea how things might shake out. I'm just a philosopher; I must rely on keener economic intellects than mine to make any prognostications. What I can say is that there is a constant demand for new content, and people are willing to offer money (and to pool their money) to pay for free content even as unsexy as PBS and NPR--or for that matter, as sexy as Wikipedia. So why don't we give it a try on a wider scale?

Why not give people a credible venue where the following scenarios could take place? It would be very low overhead simply to try it out.

IV. Some new content brokerage scenarios

  • A visual artist wants to use an obscure, more or less worthless old film that isn't yet in the public domain. He wants a way to pay for the film to be free for everyone to use. The artist offers the money for the broker to use, and the broker approaches the copyright holder and makes a credible case that the work will always be free. The copyright holder figures he'll never make any more money from the film, and agrees.
  • Cleveland Municipal School District wants to offer $100,000 to the person or group of people who produces an 8th grade general science textbook that is (1) released under an open content license, and (2) meets Ohio and Cleveland school standards. The school district approaches the content broker, which posts a call for proposals, chooses the best proposal, gives the writers some money up front, vets the result for quality and consistency with standards, acts as an intermediary between Cleveland schools and the writers, and finally publishes the textbook online and hands the (balance of the) money to the writers. Teenagers around the world can use a new professionally-written text for free.
  • A music aficionado wants to help digitize and release some of the holdings of a major folk music archive, but the archive says that the rights still rest with the family of the musicians. The aficionado gets together with his friends, who persuade a music organization to collect a $50,000 fund for the families of the musicians. The brokerage tracks down the family members, persuades them to release the music under a Creative Commons license, some of the archive holdings are digitized and made freely available online, and everybody's happy.
  • Britney Spears' fan club wants to raise $1,000,000 for Britney to record and release a new song to the public for free. Maybe they vote on a general theme or style. The fan club goes to the broker, which approaches Britney's "people," and the deal is made. The broker then publicizes the effort, saying that your credit card won't be charged until the required amount is reached. When it is reached, 24 hours later, the song plays constantly everywhere. Variant: the same thing is done but for an already-published song. Fan club says: "Let's collect $5,000,000 for ‘Oops I did it again'!"
  • The Catholic Church wants the best possible generally-accessible essay in defense of the "right to life." It hits its members up for cash and collects an astounding $2,000,000 prize. The Church then asks a content brokerage group to manage a contest: half of the money, $1,000,000, goes to the best 25-page popular defense of the "right to life." (You can easily imagine Planned Parenthood doing something very similar.)
  • I greatly admire the work of a certain philosopher. I would love for him to write an article addressed to a specific, recondite philosophical question, but I do not want him to know that it was I who asked or paid for the essay. Suppose I set $1,000 aside and ask a content broker to approach the person and make the deal. The philosopher writes the essay and publishes it publicly, saying that it was commissioned by a generous anonymous donor.
  • A major benefactor loves the idea of free, collaboratively-developed, up-to-date information--but wants the world to have something more authoritative than Wikipedia. He puts $50,000,000 into an escrow account for an expert-authored, collaborative, free encyclopedia, and says that when others match his donation, the total will be released to fund the effort. When others do match his donation, his foundation then uses part of the money to get a stellar group of academics and professionals together to spearhead the effort.
  • A publisher who wants to try out the new style of content brokerage publicizes a new offer: "Stephen King will release a new novel under a free license if enough fans pony up the required fee. Your credit card will not be charged unless enough money is actually produced." (Back in 2000, King actually did something similar to this with his unfinished project, The Plant--but without the result being free.) The conditional pledges roll in at a breakneck speed, and when the magic number is reached, all the accounts are charged and King is then obliged to release the novel to everyone, for free.

I've only started to explore the possibilities above. In fact, let's just say I've deliberately left out some very exciting possibilities. Many different kinds of media are possible to commission; many different kinds of buyers can be organized; many different kinds of content creators can be solicited (from specific people, to indefinite collaborations, to companies, etc.); many different kinds of brokerage services, from minimal to very involved.

V. The ideal content brokerage system?

One reason, perhaps, that we as a society are not commissioning more free content (apart from Public Broadcasting and Google Answers) is that we do not have a credible, visible content brokerage system in place.

There are many permutations of the general concept of content brokerage as described in sections III and IV above. If the idea is feasible and worthwhile, there might eventually be books written about how content brokerage is best managed. But I would like to make a first, amateur attempt to articulate what the ideal content brokerage system would look like. Here are some ideas.

(1) Presumably, buyers would have some incentive to employ professional content brokers--which would require that content brokers take a percentage of the amount offered, in order to pay for such things as editors and designers. Surely, if the concept becomes popular, there will be free, commissionless (and serviceless) content brokerage websites, but they will probably not be as good as those that manage the editorial and legal aspects of the work professionally.

(2) A "full-service" content brokerage would employ people who can manage all different kinds of content requests. When mature, it should employ editors and content experts who have large networks--who can quickly and reliably put their finger on the best person for commissioned jobs.

(3) The job of publisher would no doubt change considerably. Some jobs might be eliminated; new jobs would be created. But the core competencies would probably remain the same: working with authors and artists, finding people to pursue a project, building networks, design, and so forth.

(4) Content brokers should (and indeed have a business reason to) quickly develop industry standards with regard to the licensing, archiving, accessing, search, and presentation of free content. Much work remains to be done in this direction.

(5) The original copyright holder licenses the content, not the content broker. There is no need, and questionable business ethics, behind the notion of a content broker collecting much copyright itself. Its main legal role is, rather, to ensure that the terms of the exchange (money for content released under a genuine free license) are legal.

(6) Editorial decisions as to whom to award contracts should be constrained by enforceable codes of ethics. Decisionmaking should be transparent, well-documented, and easily reviewable (although not necessarily public).

(7) Communicators and artists should not be expected to work for free, or under the threat that work they spend significant time on will not be paid for. Money for commissioned work (i.e., which has not yet been created) should be placed into an escrow account, or in some other way "frozen." Contracts should make clear exactly how and under what circumstances a buyer may ask for his money back.

Of course, people might forego brokers altogether: they might approach each others with offers of money for work delivered into the hands of the general public.

VI. Some advantages and disadvantages of the new brokerage system

The primary advantage of the new brokerage system envisioned here is that content creators and support professionals continue to get paid, even when their work is free to everyone to read and use.

One disadvantage that does not exist--though one might think at first that it does--is that there is some risk about all this. The infrastructure to manage the new style of content brokerage already exists at many publishers, Internet, and media companies. Such a company--or for that matter, a university, thinktank, or foundation--could easily circulate calls for content funding with little overhead. (O'Reilly has already done something vaguely similar with its Open Books initiative.) If no one bites, so much the worse for this idea. But I suspect that there are many school districts, philanthropists, and many others, to say nothing of the ordinary people who regularly contribute to charities, who might find the idea very compelling.

One disadvantage, and it might be a deep one, is that if this new way of publishing were to win a hegemony, it would be difficult for unknown writers and artists to gain recognition except through publishing a lot without payment. But, under this new scheme, it would become de rigeur for artists to have websites in which people can access their work and where people can pay them for work already done. It would be like tipping--and that's only polite.

The ramifications are difficult to calculate; I leave further discussion of them to others or for later. What does seem clear is that it is worth thinking seriously about doing this. If it works in general, it will work in a thousand different ways.


Why Wikipedia Must Jettison Its Anti-Elitism

Wikipedia has started to hit the big time. Accordingly, several critical articles have come out, including "The Faith-Based Encyclopedia" by a former editor-in-chief of Britannica and a very widely-syndicated AP article that was given such titles as "When Information Access Is So Easy, Truth Can Be Elusive".

These articles are written by people who appear not to appreciate the merits of Wikipedia fully. I do, however; I co-founded Wikipedia. (I have since left the project.)

Wikipedia does have two big problems, and attention to them is long overdue. These problems could be eliminated by eliminating a single root problem. If the project's managers are not willing to solve it, I fear a fork (a new edition under new management, for the non-techies reading this) will probably be necessary.

Let me preface this by saying that I know Wikipedia is very cool. A lot of people do not think so, but of course they are wrong. So the following must be taken in the spirit of someone who knows and supports the mission and broad policy outlines of Wikipedia very well.

First problem: lack of public perception of credibility, particularly in areas of detail. The problem I would like to point out is not that Wikipedia is unreliable. The alleged unreliability of Wikipedia is something that the above (TechCentralStation and AP) articles make much of, but that is not my point, and I am not interested in discussing that point per se.

My point is that, regardless of whether Wikipedia actually is more or less reliable than the average encyclopedia, it is not perceived as adequately reliable by many librarians, teachers, and academics. The reason for this is not far to seek: those librarians etc. note that anybody can contribute and that there are no traditional review processes. You might hasten to reply that it does work nonetheless, and I would agree with you to a large extent, but your assurances will not put this concern to rest.

You might maintain that people are already using Wikipedia a lot, and that that implies a great deal of trust. This is true, as far as it goes; but people use many sources that they themselves believe to be unreliable, via Google searches, for example. (I do so all the time, though perhaps I shouldn't.) Perhaps Wikipedia is better described as one of those sources regarded as unreliable which people read anyway. And in this case, one might say, there's no problem: Wikipedia is being read, and it is of minimally adequate and increasing reliability. What more could you ask? In other words, why does a perception of unreliability matter?

I am willing to grant much of this reply. I think merely that there are a great many benefits that accrue from robust credibility to the public. One benefit, but only one, is support and participation by academia. I am on the academic job market now and I felt it was necessary to explain my views about Wikipedia's credibility for potential employers. A great many of my colleagues are not at all impressed with the project; but more about that in a bit.

Another benefit accruing from robust public credibility is even more widespread use and support by teachers, schools, libraries, and the general public--precisely the people who want to use what they believe to be a credible encyclopedia. To the extent that the project is not reaching, and being supported by, these people, it is not succeeding as well as it might.

Perhaps you might also maintain that, while Wikipedia does not now have a reputation for reliability, it will eventually, once enough studies proving its reliability are done, and once people are more familiar with the concept behind the project. This is hard to argue with; but it is also hard to support, because it involves predicting the future, and the future, when it comes to public opinion, is extremely unpredictable. It would be better to do something to help guarantee a reputation for reliability.

Wikipedia has another sort of credibility problem, mentioned in passing above, and I fear that time is not a solution to this problem, the way it might be to the foregoing one. Namely, one can make a good case that, when it comes to relatively specialized topics (outside of the interests of most of the contributors), the project's credibility is very uneven. If the project was lucky enough to have a writer or two well-informed about some specialized subject, and if their work was not degraded in quality by the majority of people, whose knowledge of the subject is based on paragraphs in books and mere mentions in college classes, then there might be a good, credible article on that specialized subject. Otherwise, there will be no article at all, a very amateurish-sounding article, or an article that looks like it might once have been pretty good, but which has been hacked to bits by hoi polloi. (Am I sounding elitist enough for you yet? Just wait.) One has only to compare the excellent Stanford Encyclopedia of Philosophy or The Internet Encyclopedia of Philosophy to Wikipedia's Philosophy section. From the point of view of a specialist, let's just say that Wikipedia needs a lot of work.

Second problem: the dominance of difficult people, trolls, and their enablers. I stopped participating in Wikipedia when funding for my position ran out. That does not mean that I am merely mercenary; I might have continued to participate, were it not for a certain poisonous social or political atmosphere in the project.

There are many ways to explain this problem, and I will start with just one. Far too much credence and respect accorded to people who in other Internet contexts would be labelled "trolls." There is a certain mindset associated with unmoderated Usenet groups and mailing lists that infects the collectively-managed Wikipedia project: if you react strongly to trolling, that reflects poorly on you, not (necessarily) on the troll. If you attempt to take trolls to task or demand that something be done about constant disruption by trollish behavior, the other listmembers will cry "censorship," attack you, and even come to the defense of the troll. This drama has played out thousands of times over the years on unmoderated Internet groups, and since about the fall of 2001 on the unmoderated Wikipedia.

Wikipedia has, to its credit, done something about the most serious trolling and other kinds of abuse: there is an Arbitration Committee that provides a process whereby the most disruptive users of Wikipedia can be ejected from the project.

But there are myriad abuses and problems that never make it to mediation, let alone arbitration. A few of the project's participants can be, not to put a nice word on it, pretty nasty. And this is tolerated. So, for any person who can and wants to work politely with well-meaning, rational, reasonably well-informed people--which is to say, to be sure, most people working on Wikipedia--the constant fighting can be so off-putting as to drive them away from the project. This explains why I am gone; it also explains why many others, including some extremely knowledgeable and helpful people, have left the project.

The root problem: anti-elitism, or lack of respect for expertise. There is a deeper problem--or I, at least, regard it as a problem--which explains both of the above-elaborated problems. Namely, as a community, Wikipedia lacks the habit or tradition of respect for expertise. As a community, far from being elitist (which would, in this context, mean excluding the unwashed masses), it is anti-elitist (which, in this context, means that expertise is not accorded any special respect, and snubs and disrespect of expertise is tolerated). This is one of my failures: a policy that I attempted to institute in Wikipedia's first year, but for which I did not muster adequate support, was the policy of respecting and deferring politely to experts. (Those who were there will, I hope, remember that I tried very hard.)

I need not recount the history of how this nascent policy eventually withered and died. Ultimately, it became very clear that the most active and influential members of the project--beginning with Jimmy Wales, who hired me to start a free encyclopedia project and who now manages Wikipedia and Wikimedia--were decidedly anti-elitist in the above-described sense.

Consequently, nearly everyone with much expertise but little patience will avoid editing Wikipedia, because they will--at least if they are editing articles on articles that are subject to any sort of controversy--be forced to defend their edits on article discussion pages against attacks by nonexperts. This is not perhaps so bad in itself. But if the expert should have the gall to complain to the community about the problem, he or she will be shouted down (at worst) or politely asked to "work with" persons who have proven themselves to be unreasonable (at best).

This lack of respect for expertise explains the first problem, because if the project participants had greater respect for expertise, they would have long since invited a board of academics and researchers to manage a culled version of Wikipedia (one that, I think, would not directly affect the way the main project is run). But because project participants have such a horror of the traditional deference to expertise, this sort of proposal has never been taken very seriously by most Wikipedians leading the project now. And so much the worse for Wikipedia and its reputation.

This lack of respect for expertise and authority also explains the second problem, because again if the project participants had greater respect for expertise, there would necessarily be very little patience for those who deliberately disrupt the project. This is perhaps not obvious, so let me explain. To attact and retain the participation of experts, there would have to be little patience for those who do not understand or agree with Wikipedia's mission, or even for those pretentious mediocrities who are not able to work with others constructively and recognize when there are holes in their knowledge (collectively, probably the most disruptive group of all). A less tolerant attitude toward disruption would make the project more polite, welcoming, and indeed open to the vast majority of intelligent, well-meaning people on the Internet. As it is, there are far fewer genuine experts involved in the project (though there are some, of course) than there could and should be.

It will probably be objected by some that, since I am not 100% committed to the most radical sort of openness, I do not understand why the project that I founded works: it works, I will be told, precisely because it is radically open--even anarchical.

I know, of course, that Wikipedia works because it is radically open. I recognized that as soon as anyone; indeed, it was part of the original plan. But I firmly disagree with the notion that that Wikipedia-fertilizing openness requires disrespect toward expertise. The project can both prize and praise its most knowledgeable contributors, and permit contribution by persons with no credentials whatsoever. That, in fact, was my original conception of the project. It is sad that the project did not go in that direction.

One thing that Wikipedia could do now, although I doubt that it is possible in the current atmosphere and with the current management, is to adopt an official policy of respect of and deference to expertise. Wikipedia's "key policies" have not changed since I was associated with the project; but if a policy of respect of and deference to expertise were adopted at that level, and if it were enforced somehow, perhaps the project would solve the problems described above.

But don't hold your breath. Unless there is the equivalent of a revolution in the ranks of Wikipedia, the project will not adopt this sort of policy and make it a "key policy"; or if it does, the policy will probably be not be enforced. I certainly do not expect Jimmy Wales to change his mind. I have known him since 1994 and he is a smart and thoughtful guy; I am sure he has thought through his support of radical openness and his (what I call) anti-elitism. I doubt he will change his mind about these things. And unless he does change his mind, the project itself will probably not change.

Nevertheless, everyone familiar with Wikipedia can now see the power of the basic Wikipedia idea and the crying need to get more experts on board and a publicly credible review process in place (so that there is a subset of "approved" articles--not a heavy-handed, complicated process, of course). The only way Wikipedia can achieve these things is to jettison its anti-elitism and to moderate its openness to trolls and fools; but it will almost certainly not do these things. Consequently, as Wikipedia increases in popularity and strength, I do not see how there can not be a more academic fork of the project in the future.

I hope that a university, academic consortium, or thinktank can be found to pursue a project to release vetted versions of Wikipedia articles, and I hope that the new project's managers will understand very well what has made Wikipedia work as well as it has, before they adopt any policies.