Words, words, words

Authors and writers of all stripes can learn a lot about creating and managing words from computer programmers, beginning with an appreciation for the simple, durable efficiencies of plain text. Anybody running Unix, Linux, or BSD already knows all about plain text, because it’s the third prong of the Unix Tools Philosophy:

1. Write programs that do one thing and do it well;
2. Write programs that work together;
3. Write programs to handle text streams, because that is a universal interface.

Other writers discover the efficiencies of a good text editor by chance, because they dabble in writing code, even if it’s just HTML, or they need to manage gigantic, book-length text files, without the clunky overhead of a word processor like Microsoft Word or Apple’s Pages. Unlike world processors, text editors are fast and capable of opening and editing multiple gigabyte-sized files that would cause mere word processors to choke.
True geeks prefer plain text for many reasons, but probably foremost is that plain text never changes. By that I mean, I can open the plain text files I wrote in 1981 on my first computer, a Kaypro, with a modern version of Microsoft Word, my text editor, and thousands upon thousands of other programs today.

In the Beginning Were the Words

Writers and authors who don’t use a text editor and instead use mainly word processing programs may wonder: What’s plain text, how is it a “universal interface,” and why do I care? Plain text is unformatted characters that are program independent and require little processing. Text means words, sentences, paragraphs and, yes, computer code. It’s called plain when it’s stored as unformatted, unadorned text characters in a plain text file, which sometimes has a .txt extension on the filename. A file named readme.txt probably contains plain text; if you double-click on the file icon it might even open in your text editor (probably Notepad or Wordpad if you are on Windows and have not yet downloaded a real text editor). Plain text does not light up, blink, or spontaneously create hyperlinks to Microsoft’s Bing search engine if you happen to type in a proper noun. You can’t insert your favorite YouTube video or MSNBC news item into a plain text file, although it’s easy enough to copy in the hyperlink.
Plain text means words separated by spaces; sentences separated by periods; paragraphs usually separated by single blank lines. If you are in the writing business, even the publishing or screenwriting business, it’s often all you need.

To the PC user raised on word processors, these spartan virtues sound like deficits, that is, until you want to access the text in your file using a program different than the one you used to create it. Open a Microsoft Word file in a simple text editor (like Notepad, or Textmate, or TextEdit, or BBedit or UltraEdit), and you’ll see gobbledygook, not words. Open a plain text file with almost any program, including Microsoft Word, Corel WordPerfect, Apple iWorks or any of the hundreds of text editors and word processors on any computer, and you will be able to view and edit that text, just as you could have viewed and edited it twenty or thirty years ago, just as you’ll probably be able to view and edit it twenty or thirty years from now, whether Microsoft still exists or not.

The geeks who made Unix nearly 40 years ago made plain text the universal interface because they believed in economy, simplicity, and reliability. Instead of making big complicated, bloated programs that tried to do everything (“It looks like you are writing a suicide note! Enter your zipcode or area code, and we’ll show you any local laws you may have to comply with before offing yourself.”), Unix programmers prided themselves on making small, well-designed, text-oriented programs or tools that each did one job well. One program found your files, another could open them, another could pipe the text back and forth between programs, another could count the words in a file, another could search files for matching strings of text, and so on. These programs accepted plain text as input, and produced plain text as output. Programming ingenuity meant discovering new ways to combine tools to accomplish a given task, then pass the results along (in plain text) to the next program, which could also capture, process, and produce more plain text, until you ended up with the results you sought.

During the Unix era, only an idiot would have proposed creating programs that couldn’t talk to other programs. Why would anyone create files that can be edited and viewed only by the program that created them? Say, Adobe In Design, or Microsoft Word? To Unix programmers and computer scientists the whole point was to make another tool for the Unix toolbox, then share your work with others, who in turn did likewise, and gradually Unix grew into the perfect computer geek workbench, a collection of small, efficient programs sharing a common file format and universal interface: plain text. As novelist and Uber-geek Neal Stephenson ((Author of Snowcrash, Cryptnomicon, and the Baroque Cycle trilogy of books.)) put it in his manifesto, In The Beginning Was The Command Line:

Unix … is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic . . . . What made old epics like Gilgamesh so powerful and so long-lived was that they were living bodies of narrative that many people knew by heart, and told over and over again—making their own personal embellishments whenever it struck their fancy. The bad embellishments were shouted down, the good ones picked up by others, polished, improved, and, over time, incorporated into the story. Likewise, Unix is known, loved, and understood by so many hackers that it can be re-created from scratch whenever someone needs it. This is very difficult to understand for people who are accustomed to thinking of OSes as things that absolutely have to be bought.

If Unix is the geek Gilgamesh epic, it’s a tale told in plain text. On a Unix or Linux command line, “cat readme.txt” will print the contents of readme.txt to the screen. From a Windows command line, entering the command “TYPE readme.txt” will do the same. However, if readme.doc is a Microsoft Word document, issuing the command “TYPE readme.doc” will produce a string of illegible symbols, because readme.doc is stored in a proprietary format, in this case, a Microsoft Word file.

Okay, so who cares? Most of us own a license to use Microsoft Word (on one machine, for a certain length of time), or else we can download various readers provided by Microsoft to read Word document files even if we don’t have a big honking Microsoft Word program on our computer. That’s true, for today, anyway.

But what about ten years from now? What about forty years from now? If the past is any guide, when 2035 comes around, you will not be able to open, read, and edit a Microsoft Word file that you created in 2020 and left in some remote sector of your capacious hard drive. Why? Because programs change. Companies that make proprietary programs come and go. Yes, even monster companies with the lion’s share of the word processing market. Just ask any customer of Wang Laboratories (the ruling vendor of word processors during the 1980s). Even if the company still exists, they are in the business of selling newer, bigger, more complicated, more sophisticated, and more expensive programs every other year or so. Those newer, “better,” programs come with newer, proprietary file formats, to keep you purchasing those updates.

Tales of Woe from the Elder Geeks

It takes geeks and especially geek writers of a certain age to bring home the hazards of storing information in proprietary file formats. Consider first the Seer of the Singularity himself, Ray Kurzweil, as he looks back over almost forty years of his love affair with technology and the data formats he has accumulated along the way.

In a plaintive, downright sad section of his otherwise generally upbeat take on the future of technology, Kurzweil includes a subsection called “The Longevity of Information” in a chapter of his Singularity book: How to access the data contained on a circa 1960 IBM tape drive or a Data General Nova I circa 1973?

First, Kurzweil explains, you need to find the old equipment and hope it still works. Then you need software and an operating system to run it. Are those still around somewhere? What about tech support, he asks? Hah! You can’t get a help desk worker to call you back about the latest glitch running Microsoft Office much less a program from forty years ago. “Even at the Computer History Museum most of the devices on display stopped functioning many years ago.” ((The Singularity Is Near, p. 327))

Kurzweil uses his own archival horror stories as “a microcosm of the exponentially expanding knowledge base that human civilization is accumulating,” then asks the terrible question: What if we are “writing” all of this knowledge in disappearing ink? The upshot of Kurzweil’s elegy to lost data is: “Information lasts only so long as someone cares about it.”

Do you care about your writings? The first order of business is to back up! As Kurzweil sees it, the only way data will remain alive and accessible “is if it is continually upgraded and ported to the latest hardware and software standards.” That’s one way to do it. Another way is to try to use formats that don’t go out of style. I have a 1983 Kaypro computer down in the basement that still works. All of the files I created with text editors or converted to plain text are still legible and formatted just as I left them. Here in 2012, almost 30 years after I created them, I can open them in a different text editor and work on them. The files I created using Wordstar, a proprietary word processor, are lost.

Consider another elegy to a lost file, reprinted here with permission from Neal Stephenson:

I began using Microsoft Word as soon as the first version was released around 1985. After some initial hassles I found it to be a better tool than MacWrite, which was its only competition at the time. I wrote a lot of stuff in early versions of Word, storing it all on floppies, and transferred the contents of all my floppies to my first hard drive, which I acquired around 1987. As new versions of Word came out I faithfully upgraded, reasoning that as a writer it made sense for me to spend a certain amount of money on tools.

Sometime in the mid-1980’s I attempted to open one of my old, circa-1985 Word documents using the version of Word then current: 6.0 It didn’t work. Word 6.0 did not recognize a document created by an earlier version of itself. By opening it as a text file, I was able to recover the sequences of letters that made up the text of the document. My words were still there. But the formatting had been run through a log chipper–the words I’d written were interrupted by spates of empty rectangular boxes and gibberish.

Now, in the context of a business (the chief market for Word) this sort of thing is only an annoyance–one of the routine hassles that go along with using computers. It’s easy to buy little file converter programs that will take care of this problem. But if you are a writer whose career is words, whose professional identity is a corpus of written documents, this kind of thing is extremely disquieting. There are very few fixed assumptions in my line of work, but one of them is that once you have written a word, it is written, and cannot be unwritten. The ink stains the paper, the chisel cuts the stone, the stylus marks the clay, and something has irrevocably happened (my brother-in-law is a theologian who reads 3,250-year-old cuneiform tablets–he can recognize the handwriting of particular scribes, and identify them by name). But word-processing software–particularly the sort that employs special, complex file formats–has the eldritch power to unwrite things. A small change in file formats, or a few twiddled bits, and months’ or years’ literary output can cease to exist.

Now this was technically a fault in the application (Word 6.0 for the Macintosh) not the operating system (MacOS 7 point something) and so the initial target of my annoyance was the people who were responsible for Word. But. On the other hand, I could have chosen the “save as text” option in Word and saved all of my documents as simple telegrams, and this problem would not have arisen. Instead I had allowed myself to be seduced by all of those flashy formatting options that hadn’t even existed until GUIs had come along to make them practicable. I had gotten into the habit of using them to make my documents look pretty (perhaps prettier than they deserved to look; all of the old documents on those floppies turned out to be more or less crap). Now I was paying the price for that self-indulgence. Technology had moved on and found ways to make my documents look even prettier, and the consequence of it was that all old ugly documents had ceased to exist. ((In The Beginning Was The Command Line. Available at Neal Stephenson’s website (reprinted here with permission from the author).))

Microsoft Word vs. Plain Text

The founding fathers of Unix did not glorify plain text only because they were worried about permanency. No, they called plain text “universal” because it’s so easy to read, scan, search, access, pipe back and forth, share. Now. Forty years ago, and forty years from now.

Plain text it is! And in true Unix fashion, the best tools for creating and managing text (text editors and file search programs) are often not the same as the best tools for presenting text for the consumption of others (word processors, LaTeX, and other document preparation programs). I hope to post two more articles: One on text editors and Unix file utilities; and another on what might be called document presentation programs: Microsoft Word, LaTeX, Final Draft and Movie Magic Screenwriter. Programs like Highland and pandoc, Markdown and Fountain, and other mark-up and conversion systems allow writers and authors to type text once, and then convert it as needed for the Internet, for print, for e-book, screenplay, or manuscript format.

(Excerpted from Rapture For The Geeks: When AI outsmarts IQ, by Richard Dooling. This is the first of three articles I plan to write on plain text, text editors and other writing tools, including Fountain and Highland for screenwriters and Markdown for novelists.)