Dreaming in Code

I began the article "How I Started a Code Study Group (and Why You Should Too)" by explaining how excited I was to start working at BigCommerce. It was only fair to summarize it in a word, or I would have needed a separate piece to express how pleased I was. Since a young age, I'd wanted to join a start-up but had little idea what that would entail. I imagined the early days of Apple or Google, where thoughts ran rampant. There are many likely culprits for why I believed this, the most influential of which was Scott Rosenberg's "Dreaming in Code:  Two Dozen Programmers, Three Years, 4732 Bugs, and One Quest for Transcendent Software".

Dreaming in code, I partook in regularly by that point in life, my mind riddled with Lua, the language of choice for homebrew PlayStation Portable (PSP) apps. I initially grabbed "Dreaming in Code" from the display shelf at my library as a teenager and couldn't believe my luck. The local branch's technical section had become incredibly outdated by web standards. "On the Road" read much differently for me at age 12 in 2005 then it may have in 1995.

To me, "Dreaming in Code" is a current technical and cultural perspective about the continued rising action of this informational age. I've recently realized how it tied together my progression as a programmer, and my favorite aspect of "Dreaming in Code" is Rosenberg's inclusion of several "programming laws." These, woven together by Rosenberg, help explain how one slippery coding bug can travel through the most secure safety nets.

Teenage Debugging

Around the same time, I had learned of the PSP hacking community when I joined a podcast's forums and later made what I consider my first "contribution" to the Internet on their "How-To" boards. I made some music and graphics for this episode, too.

The PSP group was very technical when the Internet primarily consisted of technical people. I became unconcerned with the state of any beasts in the machine. "Bricking" your handheld was standard nomenclature when reading the README file that was RARed with what you were trying to do (it was an EBOOT.PBP). In short: you relinquish your rights to the possibility that you could render your device as useful as a brick. We were the rebels of the gaming world, unsatisfied with its current offerings. Like C, C++, Python, and PHP before it, my interest in Lua waned. I found it to be very limited in its basic form. To me, being tied to a device was reckless.

For some years prior, I focused on graphic design groups like
Skandalouz Designz (Forum archive, Website archive) and mAsDesigns to learn how Adobe PhotoShop works. After these similar experiences, I noticed a recurring theme of sharing technical problems with web-based communities. Finding decent or near-complete screenshots of forums from this period is especially painful. Most free image hosts have effectively broken decades-since-abandoned accounts (Photobucket!!) and respective images. One of mAsDesigns two graphic designers, Sunjo, is sometimes active on deviantArt, and you can see what these templates looked like below:

Found on deviantArt

Found on deviantArt

What set mAsDesigns' site experience and design apart was the synergy between their solo coder Michael Wright and designers Sven and Sunjo. You can get a sense of these communities through Michael (Wrighty)'s goodbye:

For those familiar with the ProBoards world, Wrighty might be a familiar name similar to how california should be. These are honorable mentions and deserve their contribution. Similarly, this post wouldn't have been possible without my friend Keenan to bounce ideas off of and learn discrete technical details from. I want to give my biggest thanks to you for being a friend willing to entertain even my most incomprehensible of questions. In the aforementioned PSP community, we met in our early teenage years and worked on prototype games together.

The Internet had varying misgivings with browsers, but sending code to production targeting a specific device (or Operating System) made it ineffable as a trade-off. This wasn't as obvious to me then, but as quoted in "Dreaming in Code":

"All programmers are optimists," Frederick Brooks wrote in 1975. "Perhaps the hundreds of nitty frustrations drive away all but those who habitually focus on the end goal. Perhaps it is merely that computers are young, programmers are younger, and the young are always optimists." But programmers' innate optimism is often obscured by their forthrightness about difficulties and problems. Reporters know that when you can't get a straight answer from a salesperson or a marketer about a product, an engineer will (when not locked away or ordered to be silent) speak the hard truth. In my very first conversation with Chandler's original software architect, John Anderson, at a time when I understood the project to be in its infancy, he sat down across from me and, before I could ask a question, declared, "Here's what our three biggest failures are." So if programmers are optimists by nature, they also have a keen eye for the downside. A hyperactive imagination for disaster scenarios is a professional asset; they have to think through everything that can go wrong in order to practice their craft. ^[1]

Beginning work in early 2003, the OSAF programmers this volume centered around wouldn't be prepared to utilize the front-end web when iPhone shattered the computing industry four years later.

It might be a stretch for a Web-oriented platform like Mozilla, which was designed to display data in static screenful delivered from a server. A Mozilla-based Chandler would need to base such a feature on Javascript, a programming language meant specifically for embedding program code inside Web pages. (HTML, the basic language for publishing Web pages, isn't really a programming language at all; it just tells a browser what to show on screen.)

"You never see this in a Web application because the client can only make a few round trips to the server in a second," Kapor says. "I want a sense of confidence that we'll have a first-class way to do this. To me this is at the heart of the whole PC-versus-Web thing. You can have only so many round trips to the server per second or your performance gets clobbered."^[2]

It only fits the first chapter titled "Doomed."

Why Do We Build Castles in the Sky?

The Chandler project's language of choice was Python, something as flexible as my favored JavaScript was unthinkable. A lack of persistent databases like MySQL rendered it reasonably limited in programming. JavaScript's possibilities have since been sent into the stratosphere with Node.js.

Next.js, Gatsby, 11ty, Vue.js, there is a mouthful of popular Node.js-based frameworks that wax and wane faster than I learn about them. That isn't to say this is a problem. This is "The Bazaar" of goods we've been building with miniature castles in the near literal sky:

"The programmer, like the poet, works only slightly removed from pure thought-stuff," Frederick Brooks wrote. "He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures." Brooks's passage carries echoes of some of Shakespeare's best-known lines from A Midsummer Night's Dream:

The lunatic, the lover, and the poet

Are of imagination all compact. . .

The poet's eye, in a fine frenzy rolling,

Doth glance from heaven to earth, from earth to heaven;

And as imagination bodies forth

The forms of things unknown, the poet's pen

Turns them to shapes, and gives to airy nothing

A local habitation and a name.

The words belong to Theseus, the Duke of Athens, who lumps together "the lunatic, the lover, and the poet" as victims of their own imagination's flights. A devotee of "cool reason," he's contemptuous of the labor he describes—the "tricks" of "seething brains."^[3]

By the sky, I mean Linux servers, which like many of these topics, can be challenging to compact enough into a digestible form. There's an understandable indifference toward Linux. It's understood that you'll likely require troubleshooting during installation, which creates a barrier to entry for any "normal" user. For Linux, it's a near expectation your graphic card drivers might not work (meaning you won't have a display to work with). With that alone, the majority of users will be confused.

Canonical created the Ubuntu project intending to be "Linux for human beings." From here, I can delineate my core beliefs in the tech industry regarding Free and Open Source Software (FOSS). As standard with open-source projects, these ideas are open to debate.

What intrigued me the most about the OSAF group was their work method and goals. They operated under FOSS-friendly agreements while receiving corporate-level funds from OSAF's founder Mitch Kapor and set out with specific tenants in mind.

Chandler Design Goals

Build on open source software that supports open standards, choosing projects that are reliable, well documented, and widely used

Use the Python language at the top level to orchestrate low level, higher performance code

Design a platform that supports an extensible modular architecture

For the desktop client, choose a cross-platform user interface toolkit that provides native user experience

Use a persistent object database

Build in security from the ground up

Build an architecture that supports sharing, communication, and collaboration

The Chandler Project

Mitch Kapor founded Lotus Software in 1982, known for Lotus 1-2-3, an early example of the "killer app" for a platform. Focused on three activities for users of the program, Lotus 1-2-3 stood the opposite of an academic approach taken by the equally amicable VisiCalc. This spreadsheet application incorporated many rare features at the time and has since become a necessity, like macros. Another milestone, you could quickly and easily graph data when Operating Systems hadn't popularized a GUI yet. By driving VisiCalc out of existence and IBM PC sales skyward, Kapor and the original developer of Lotus 1-2-3, Jonathan Sachs, created the first killer app.

After leaving Lotus in 1986, Kapor founded the Electronic Frontier Foundation (EFF.org) and later, in 2003, helped found the Mozilla Foundation. An investor in the computing industry and related causes, he began feeling that itch all engineers at heart need to satisfy.

His motivation to create Chandler—an ambitious rethinking of PIM software to enable easy sharing of data and to run on the three most popular kinds of personal computers—was, undoubtedly, to use Raymond's phrase, "scratching a personal itch." He wanted to build Chandler because he craved something like it, and similar existing products simply didn't match the picture in his head. But he didn't begin, as a lone open source hacker might have, by posting a first chunk of code; he began, as an entrepreneur would, by subleasing some office space.^[4]

Steve Jobs took this to an extreme, considering his role on the planet, to "put a dent in the universe," Apple's hallmarks resemble the Chandler tenants of design. These can be derived from the first-mentioned law's inspiration:

"The Cathedral and the Bazaar" dissected Torvalds's style of managing the Linux project over (then) half a decade's development from personal hobby to global phenomenon and derived a set of principles from it. "Every good work of software starts by scratching a developer's personal itch. Programmers are motivated and led toward their best work by a desire to accomplish something that pleases them or fulfills a personal need. "Good programmers know what to write. Great ones know what to rewrite (and reuse)." No programmer will want to build something from scratch if he can grab something that is already written and adapt it to new ends."^[5]

Rough Translations

Software is Hard^[6]

Often in computing, we discuss "executables," "applications," and "programs" with variance, and clearly defining these abstract concepts in such minutiae may be the same defiance "programmers" face when speaking with machines. The "execution" and in-so-far-as defined by Merriam-Webster: "1: to carry out fully: put completely into effect" was a recent topic of interest with a colleague. We were attempting to define "execution" or the program's performance. I may be paraphrasing on his behalf, but we decided it's relative to the feeling of accomplishment and pride that occurs when executed to perfection.

By revisiting a time when the web started to accelerate, I found many of these laws have proven themselves. This is only exacerbated by an acquisition of the RedHat Linux distribution by IBM to the tune of $34B and a more open-ended approach by Microsoft (who also put their money where their mouth is by acquiring GitHub). In 2016, Microsoft joined the board of the Linux Foundation as a platinum member and then, in early 2020, added to their collection with npm.

Rosenberg stuck in my prototypical software developer's mind. As a teenager, this lit countless metaphorical beacons in the technical realm. It sparked an interest in Version Control, Test-Driven Design and led to my installing more than the Ubuntu distro because of these tendrils.

Chandler was a philosophical project with lofty goals and lacked direction. When eventual leaders relented to taking over management roles, decisions were made, but often too late.

After each re-read of "Dreaming in Code," I'm left with a sense of wanting. Many of Chandler's goals have been fulfilled in other ways, but the core of it remains in the ether and never realized to perfection.

In planning my project I had failed to take into account Hofstadter's Law, the recursive principle to which Douglas Hofstadter attached his name: It always takes longer than you expect, even when you take into account Hofstadter's Law. This strange loop seemed to define the essence of software time. Now I was stuck in it myself. After three years, Chandler was beginning to become a somewhat usable, though incomplete, calendar program. But I could not say with any confidence how much longer it would take for the project to deliver something like its original promise. OSAF's managers had a plan that involved four more "dot releases" over roughly two more years to arrive at a 1.0 version of the program—but the shape and scope of that program remained open to further revision and course correction.^[7]

The Chandler project has since become a scar of the Internet, made history in literature as a relic of "where dragons be," and is soon destined to become utterly inaccessible in an SSL-strict future. The software is so dated that while it brings a smile of nostalgia on my face when I open it, it's sadly due to the long-lived wxWidget flicker greeting me. Bug 44's discussed label that ended with "scary" was more accurately described as "black hole." From Eric Raymond's "The Cathedral and the Bazaar":

It's fairly clear that one cannot code from the ground up in bazaar style. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. . . . Your nascent developer community needs to have something runnable and testable to play with. When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is convince potential co-developers that it can be evolved into something really neat in the foreseeable future.^[8]

Linux

Closed-source software was uncommon until the mid-1970s to the 1980s, when IBM implemented in 1983 an "object code only" policy, no longer distributing source code. IBM's stated reason for this was users would make modifications to their source, and IBM was expected to support these changes, which was unfeasible. Nothing in open source was formalized until at MIT, Richard Stallman began the GNU (GNU is Not Unix) project in 1984, which created critical tools that formed a portion of an Open Source operating system. It was, however, missing a most vital piece: the kernel.

On most systems, it is one of the first programs loaded on start-up after the bootloader:

Computer system inventors escaped this dilemma by using a small program called a "bootstrap loader" that gave the machine just enough capabilities to load the big operating system into memory and begin normal functioning. In early generations of computing, human operators would enter the bootstrap loader manually via console switches or from punch cards; modern computers store it on fixed memory chips.^[8:1]

Summarily, it handles the rest of the start-up and input/output requests from software, translating them into data-processing instructions for the central processing unit. It manages memory and peripherals like keyboards, monitors, printers, and speakers. You should check out systemd and related discussions, for those interested in more technical details.

In 1991, Linus Torvalds, who at the time was a second-year graduate student at the University of Helsinki, wrote and distributed a Unix-like kernel (what's Unix?). In the first noted manner of something like FOSS development, it was distributed widely, improved upon, and soon adapted to become the core of the GNU/Linux operating system. This glosses over how vital each of these projects has been in their respective ways, but I encourage anyone to share their thoughts in the comments since it's a bit opinionated ;).

The Cathedral and the Bazaar

Used in an essay written by Eric Raymond comparing two models of software, this concept is instrumental for understanding the relationship between FOSS and Linux:

In the Cathedral model, source code is available with each software release, but code developed between releases is restricted to an exclusive group of software developers. GNU Emacs and GCC were presented as examples.

The Bazaar model, in which the code is developed over the Internet in view of the public. Raymond credits Linus Torvalds, leader of the Linux kernel project, as the inventor of this process. Raymond also provides anecdotal accounts of implementing this model for the Fetchmail project.

In full.

Abstract:

I anatomize a successful open-source project, fetchmail, that was run as a deliberate test of some surprising theories about software engineering suggested by the history of Linux. I discuss these theories in terms of two fundamentally different development styles, the "cathedral" model of most of the commercial world versus the "bazaar" model of the Linux world. I show that these models derive from opposing assumptions about the nature of the software-debugging task. I then make a sustained argument from the Linux experience for the proposition that "Given enough eyeballs, all bugs are shallow," suggest productive analogies with other self-correcting systems of selfish agents, and conclude with some exploration of the implications of this insight for the future of software.

While self-contained, Rosenberg offers additional insight within the context of the Chandler project:

Maybe Eric Raymond's "The Cathedral and the Bazaar" had been wrong, and Linus's Law ("'Given enough eyeballs, all bugs are shallow") didn't transcend Brooks's Law after all. Or perhaps OSAF, for all its transparency, had so far failed to meet Raymond's requirement for success with a bazaar-style open source project—that it must recognize, embrace, and reward good ideas from outsiders. Richard Stallman, the godfather of free software, liked to say, "When people ask me when something will be finished, I respond, 'It will be ready sooner if you help.'" OSAF welcomed volunteers and external contributions, but Chandler's grand design ambitions and sluggish pace of delivery had made it hard for outsiders to pitch in.^[9]

Distributions

Packaging and distributing your operating system running the Linux kernel is commonplace. There are virtually limitless "distros" available, with more being created daily. They vary in ease of use and technical capability and are all reasonably customizable (some more than others). Based on the popular Ubuntu, I recommend starting with Linux Mint,  but I reiterate: this is open to debate.

For an up-to-date list of the most popular distros, see distrowatch here. Install any distro at your own risk. The live mode is an excellent way to try out an OS. It is also much slower than running through your local disk.

A similar concept is repositories ("repos"). You might have seen repositories in places like GitHub, but consider Apple's App Store. Excluded initially from iPhone OS (today, iOS), the only way to retrieve external applications on the original iPhone was by "jailbreaking" your device to install the Cydia app. Cydia offered what would be seen on any other Unix device: repository sources. It was only after what Cydia did in February 2008 that Apple launched its walled-garden iteration less than six months later in July. The same concept has since come into fruition on macOS and Windows.

The Command Line

While it isn't necessary, a correlation can be drawn between users using Linux and those using the command line. If you've used the command prompt in Windows, it's a similar concept. You enter commands via typed commands into a virtual terminal. This process allows you to communicate directly to the kernel and issue commands that (for better or worse) give you absolute control over the Operating System.

An excellent resource for learning more about the Linux Command Line Interface is "The Linux Command Line" by William E. Shotts, Jr.

Lucid Dreaming

Another cog in a continuous pattern of finding a means to an end, the very end didn't have a destination beyond knowledge from a subconscious standpoint attempting to sponge what all of this meant. In an exploded view, my impatience (eagerness) is laughable (incredible). If I had known then what I know now, I might've told myself to turn around, but my stubborn (persistent) self would ignore it anyway.

I've felt my engineering itch Raymond refers to. The web feels stale. When I traverse the Internet today, I'm not in the same awe I was in 2007. I'm not sure when this shifted, but it's as though we've pressed against our castle design's ceiling. Castle towers can only be built so high.

We've become inundated with gestures over substance when we need continuity of an abundant source, but one so declarative that it can be uniformly agreed upon. This isn't an easy task, but one challenged by the all-exalted Google. Any fortune teller could lead us down where that path leads. I think a lot of the answers we seek aren't down ones typically followed.

These should be the same philosophical questions we asked ourselves as we turned the millennium and engaged with approach rather than the result, or we'll be lost to these questions forever. As ubiquity reigns, all leaders be supreme, but less so of the people.

Time is hard.

Being a person of process, I find that by enjoying the process in itself, I'll lose whether it's something to lose myself in, and it's from my network of rabbit holes in code that I can echo back an error in the pipeline. For Chandler, Kapor wanted the "soul of Agenda," something that can only be summarized by Kapor himself:

But he never really moved on from Agenda. He remained, he'd say, "so emotionally attached to this product, it's like my child." As he began winding down his career as a venture capitalist, he contracted with some programmers to build an up-to-date version of Agenda using Java. But the result was unsatisfying. "It felt narrow, old-fashioned," Kapor says. "It was kind of like building a model-T Ford. We know a lot more about cars now!" It turned out that what he cherished was not the "feature set" of Agenda —- the list of specific things it could do —- but the program's spirit of dynamic flexibility, of "put it in first, make decisions later." Whatever shape his new software would take, Kapor decided, as he pondered the inadequacies of
Microsoft Exchange and began to dream of inventing something to put in its place, it would have to conjure the soul of Agenda.^[10]

So while we sit on our castles upon high, we should remember some of our more agreeable rules in design.

In 1990, at the PC Forum gathering of computer industry luminaries, Kapor first delivered the text of his "Software Design Manifesto."

No one is speaking for the poor user. There is a conspiracy

of silence on this issue. . . . Scratch the surface and you'll

find that people are embarrassed to say they find these

devices hard to use. They think the fault is their own. . . .

Everyone I know (including me) feels the urge to throw

that infuriating machine through the window at least once

a week. . . . The lack of usability of software and the poor

design of programs are the secret shame of the industry. . . .

What is to be done? Computing professionals themselves

should take responsibility for creating a positive user experience.

Perhaps the most important conceptual move to be

taken is to recognize the critical role of design, as a counterpart

to programming, in the creation of computer artifacts.

And the most important social evolution within the computing

professions would be to create a role for the software

designer as a champion of the user experience.

Software design, Kapor argued, was not simply a matter of hanging attractive graphics atop the programmers' code. It was the fundamental creative act of imagining the user's needs and devising structures in software to fulfill those needs. As the architect draws up blueprints for the construction team, so the software designer should create the floor plans and
elevation views from which software engineers would work. Reaching back to ancient Rome, Kapor proposed applying to software the architecture theorist Vitruvius's principles of good design: firmness—sound structure, no bugs; commodity—"A program should be suitable for the purposes for which it was intended"; delight—"The experience of using the program should be a pleasurable one."^[11]

Conclusion

I hope this can serve as its signal in the same philosophical capacity Chandler aimed to reach for, or at least served for me. Chandler embraced Open Source and shared their code but lacked the documentation for anyone to contribute. I question the whole approach Chandler made in its strides to the soul of Agenda, this concept I have no basis on but feel some responsibility for. While they understood the base of a level to some degree of what they were trying to discover, I'm unsure if it was proper to that desired core. Agenda handles time.

Time is incremental but definable. It is a center of control (by culture) but can also be de-centralized by redefinition. Digression is best suited for such a suggestion, as it still carries even more majestic ideas than Chandler's.

Easter Eggs

For the astute, many of these links lead to insecure pages, an unfortunate event because I knew the insecure Chandler links likely meant that there wasn't anyone actively maintaining Chandler. We can only do so much to preserve the software and the knowledge and unique events surrounding it. There are hidden events called Easter Eggs in software, and they have become more widely known due to their inclusion in video games. Think of this as an easter egg, like a colophon, or just another piece of the static you see.

"We got here by having high goals," Toy continues. "Part of our struggle is in not valuing the goals equally. But there's no disagreement over the list itself. The biggest fear is that the goals are simply not meetable. We do have a proposed design. It's got some holes. It's time to talk about those holes and make sure we have plans to fill those holes so that never again, as long as I live, do we roll our eyes when the word data comes out of our mouths. This is the snake that must be killed."

"And in this meeting," Hertzfeld says, "we kill the snake—we don't just make plans to kill the snake."

"Yes," Toy answers.

"The dragon has many swords in it."

"There's also some bodies of brave knights who walked into the flames."^[12]

"Dreaming in Code" pp.54 ↩︎

"Dreaming in Code" pp.157 ↩︎

"Dreaming in Code" pp.64 ↩︎

"Dreaming in Code" pp.27-28 ↩︎

"Dreaming in Code" pp.23-24 ↩︎

Donald Knuth, http://www.ams.org/notices/200203/fea-knuth.pdf ↩︎

"Dreaming in Code" pp. 331 ↩︎

"The Cathedral and the Bazaar" http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ ↩︎ ↩︎

"Dreaming in Code" pp.44 ↩︎

"Dreaming in Code" pp.47 ↩︎

"Dreaming in Code" pp.149 ↩︎

"Dreaming in Code" pp.109 ↩︎