The recent publication of XML 1.0 Fifth Edition, and the ensuing discussion of how best to define the set of name characters, has made me think about a proposal that came up during the original development of XML 1.0.

We had taken as our task the definition of a grammar production to select all and only the Unicode characters suitable for use in identifiers. For our SGML-influenced minds, suitability essentially meant being a letter or something letter-like, by analogy with the characters A through Z and a through z allowed in SGML’s reference concrete syntax, or a numeric symbol, by analogy with 0 through 9. Now, it is an empirical fact that any such grammar production will be complicated, especially if it excludes not just unsuitable Unicode characters but unassigned code points.

As we were contemplating the unappetizing lists of ranges and properties that became productions [84] through [89] of the XML spec, someone — my memory says it was Tim Bray, but I have not tried to verify this memory against the historical record, and I don’t remember whether it was in email or on a phone call — said “Well, I realize I risk being labeled a xenophobe for suggesting this, but you know, the lexer has to have character tables for recognizing names and other tokens, and this rule is going to make those character tables just huge. We all know that in the ideal, normal case, the user is not going to be eyeballing XML source directly, you’re going to want to have some sort of presentation interface. So the element names are just codes, not utterances in natural language. They might as well be E001, E002, E003, and so on, as far as the parser is concerned. So why don’t we just do the same thing as SGML’s reference concrete syntax and restrict names to ASCII characters? That would make the tables you need for the lexical scanner a lot smaller. Any UCS character can go in the data, and the data is all the human reader needs to care about anyway. It’s only in the element and attribute names that we make the restriction, and the proposal is not really Anglo-centric since element and attribute names don’t have to be natural-language words; they shouldn’t be part of the user interface in the first place.”

I remember having a sinking heart at this point. The analysis is so plausible, and yet wrong in so many ways. One of the design goals of XML is that it should be legible to humans, and having identifiers that take the form of words in a natural language is an important tool in making markup legible. E001, E002, and so on just don’t cut it. (A librarian did suggest once in all seriousness that SGML vocabularies should have numeric identifiers, like the numeric field identifiers in MARC records. 245 means main title, everyone knows that, but unlike “main title” it doesn’t have an Anglophone bias. But as a design principle for actual vocabularies, this seemed to me too much like ensuring that all languages are treated equally by ensuring that the vocabulary is hard to understand for everyone.)

Also, while I have seen some very nice interfaces to marked up text, I’ve spent most of my writing life over the last twenty years working with SGML and XML source, mostly in emacs, and no interface that hides the markup has ever made me want to change. (For a while I did use the hide-tags mode in XMetal for rereading and copy editing, but I could never draft in it, and when I upgraded my operating system and XMetal ceased to run, I didn’t actually spend a lot of time looking for a new copy, or a new GUI.) I don’t want to hide the tags. The tags are what make things work. There is a story that a worker came into Bertolt Brecht’s apartment in Berlin once to install some curtains, and after doing so he started to put up a valance to hide the curtain rod and the hooks and strings. Brecht came into the room and made him stop and take it back down, with (as I remember it — I have not spent any time looking for the source of this story to make sure I’m getting the details right) the words “Never hide the mechanism!” When it comes to markup, at least, I’m with Brecht.

So for me, and anyone like me, the claim “no human really ever looks at this stuff” is false (and also conveys indirectly the suggestion that we counter-examples aren’t really human and thus don’t need to be accounted for). Even if everyone in the world used tag-hiding editors, the programmers who create those editors and the stylesheet authors who specify the mapping from tags to display form will spend time looking at the raw markup. And last I looked, they were all humans, too.

But none of those arguments were going to go anywhere against that brutally simple argument about the size and complexity of the scanner tables, and the seductive suggestion that it really would be inhumane to require users to use tag-visible interfaces. So for a while I was really worried that the proposal for ASCII-only identifiers was going to carry the day.

I’m not sure whether my memory of what happened next is what happened in reality, or whether I’ve just persuaded myself to accept as real a fantasy of what could have happened, if only someone had had the wit to think of it, and the nerve to say it, at the time.

The way I remember it is this. Someone (I don’t remember who) responds to the suggestion carefully, thoughtfully, and without screaming. They agree that it’s important to keep the lexical scanner simple, and that identifiers really don’t need to be used to represent arbitrary natural-language words or phrases. And they conclude by saying “So I think I’m willing to support this proposal, if we can improve it a little bit. Right now, the natural way to represent the scanner table for ASCII-only identifiers is to use a table 128 octets wide. But that’s a lot bigger than we need. So I propose that we don’t allow all the characters A through Z in both upper and lower case. If we restrict ourselves to A through O, uppercase only, then we can get by with a much smaller table. So I’ll support the proposal if we restrict identifiers not to any ASCII letter, but to A through O uppercase only, i.e. U+0041 through U+004F. And since no one wants to use natural-language words for identifiers, that’s not a problem, right?”

And then, silence. You can just about hear the proponents of ASCII-only identifiers starting to say “but then you couldn’t use ‘q” or ‘quote” or even ‘html” or ‘body” as element names,” before biting their tongue as they realize that they can’t say that without giving away the store.

Whether it happened the way I remember it or in some other less dramatic and memorable way, the end is the same: after people thought about it for a while, the proposal for ASCII-only identifiers evaporated, like dew in bright sunlight.

Tim Bray’s prose sketch of Jon Bosak is good, and vivid, but it doesn’t mention what I think is one of Jon’s outstanding traits. In a quiet, utterly unassuming way, Jon is one of the most persuasive and politically astute people I have ever met. He will not thank me for pointing this out: I think he thinks that if people know, they’ll be on their guard. He doesn’t do a hard sell (at least not to me); he takes the trouble to understand where his interlocutor is coming from and to find common ground with them. And he has patience; he is not dissuaded from a goal by the idea that it might take a while, or that it must be approached indirectly. And he is very reticent about taking credit.

We wrote Jon’s name into the XML spec, in the passage

XML was developed by an XML Working Group … formed under the auspices of the World Wide Web Consortium (W3C) in 1996. It was chaired by Jon Bosak of Sun Microsystems with the active participation of an XML Special Interest Group … also organized by the W3C.

because we wanted to get Jon’s contribution on record and force him to accept credit. Without Tim Bray, or without me, or any of the other members of the editorial review board and working group, the spec would have been different. Without Jon it would not have come to pass.

In memory of a particularly difficult political task undertaken and successfully negotiated, some of Jon’s friends once gave him a gift that I have always thought apposite: a dark gray bomber jacket, embroidered in dark gray (so the embroidery was virtually invisible) with the words “Subtle and devious”.

Hail, xml:Father!

[11 February 2008]

The XML spec became a W3C Recommendation ten years ago this week.

Tim Bray has posted some character sketches from the period; Eve Maler has followed suit with some recollections (and an online version of Maler/El Andaloussi! Woo hoo!); this has inspired me to think about doing the same. What follows is the first in (what I hope will be) a series of moments I remember from the creation of XML.

If you look, you can find a lot of stories about the beginning of XML. It surprised me, at first, that they all seem to be different; it surprised me even more to find some told in the first person by people whom I had not suspected of being involved with XML at all. But I shouldn’t have been surprised. Scores or hundreds of people were involved in the development of XML, thousands in its spread and uptake. In some sense, then, XML will have had scores, or hundreds, or thousands of beginnings. Why should I think I know about them all? Questions like “How did X start?” often mean not “How did X start?” but “How did you come to be involved in X?” — or, at least, that’s how we answer them. The beginnings of XML? I don’t know. But I’ll tell you what I do know; I know when I first heard about it.

The second WWW conference was in Chicago, in October 1994. With Bob Goldstein, one of my colleagues at the University of Illinois at Chicago computer center, I had submitted a paper on how the Web would achieve its true potential only once it had SGML awareness (“HTML to the Max”). We had to do it for the conference in Chicago, because the computer center would certainly not have paid for travel to go anywhere else; they weren’t happy at being asked to pay the registration fee, let alone travel.

This was the conference at which I saw SoftQuad Panorama for the first time, but I could not figure out the stylesheet mechanism well enough to use it to deliver my slides; later I did, and used Panorama for slides for years after that.

Sometime during the conference I ran into Jon Bosak, whom I knew from SGML conferences, and Dave Hollander, whom I may have been meeting for the first time. Jon was at Novell, Dave was at HP (where I believe he was in charge of building hp.com), and the conference hotel (as I remember it, the Hilton on Michigan Avenue it) was hopelessly overcrowded; there was no chance of finding a place to sit and have a cup of coffee.

Fortunately, I was local, and knew the neighborhood slightly, so I led Jon and Dave up the street a bit to the Cafe des Artistes, where we sat on the sidewalk in the October sun and talked about markup and the Web. Browser makers didn’t want to support SGML; they thought it was too complicated and would make the browsers heavy-weight. This seemed like an excuse to me: browsers were already putting on more weight than SGML support would have required. We speculated about defining a simpler profile of SGML, so they couldn’t use complexity as an excuse. Where? The ISO working group was not going much of anywhere with the five-year review and revision of the spec; it seemed unlikely anyone would get any joy there. SGML Open? Somewhere else? Jon had an idea it might fly as an idea for a working group at the new World Wide Web Consortium. Dave thought that SGML support would be the best thing, but if we couldn’t persuade people to do that, he had a fallback idea. HTML appeared hopelessly unequal to the task of publishing serious technical documentation, but Dave’s experience developing the Semantic Delivery Language (SDL) had persuaded him that you could have a reasonably small markup language with some hooks for richer semantics. SDL had been developed for help systems and delivery on CD-ROMs, and in it each element carried an attribute that said what kind of element it had come from in the SGML vocabulary in which the document had been prepared. (Nowadays you’d do that with the HTML class element.)

[Googling for “semantic delivery language” just now, I find that about that time Jon put together a proposal for specifying a Hypertext Delivery Language (HDL) based on SDL and optimized for WWW delivery; the copy on the W3C site says it was last modified 1 November 1994, less than a month after the Web conference.]

Other people had specified subsets of SGML before; I had done one myself in 1992 (“Poor-Folks SGML”), and several others were also used as input to the design of XML. And of course I had just given a paper urging that Web software start supporting SGML.

But I think of that cup of coffee at the Cafe des Artistes on Michigan Avenue as the first time I heard anyone talking about the specific idea that became XML.

And you, dear reader? What are your recollections of the beginnings of XML? When did you start thinking about the ideas that became XML? When did you first encounter the effort that in the end produced the spec? Tell me here, or write about it on your own blog.