Structure and service: Illuminations from a trip to the forest

By Andy Oram

January 31, 2003

This article was originally published on the O’Reilly Media web site.

My idea of a pleasant hike involves a few hours of ambulation through a National Park. It was certainly not personal choice that led me over the mountains of Northern Oregon and down a virtually uncharted patch of an eastward-looking gulch through guantlets of towering Douglas fir trees to the slavering mud near a stream whose rushing sound filled the valley.

My mission had started a couple weeks before, when the O’Reilly Conferences group started planning the second Emerging Technology conference, scheduled for April of this year. In the hope of finding ways to drum up new attendees, I had met with organizers Rael Dornfest, Gina Blaber, and Nat Torkington.

“Why don’t we jostle our regulars and have some fun by seasoning the conference with a really different population?” Rael started as we plopped ourselves down around the sleek, eliptical table in our recently built conference room.

“Do you want to move beyond the Silicon Valley crowd?,” I asked. “Think about the Pacific Northwest. You’re locating the Open Source Conference there this year, so why not explore it?”

“Excellent idea!” cried Gina. “Who would you like to visit?”

“Besides the obvious choice of Microsoft, there are Intel facilities in that region.” I continued throwing out names in debonair disregard for the staccato of obscene comments that Nat made following the mention of each company. “LSI Logic has a facility, I think, and there’s a compiler company called The Portland Group—”

“Oh, Andy!” groaned Gina. “I was expecting something much more creative from you. Those companies are fine, but so old.”

“We wanted you to scare up something a bit on the outrageous side of innovation,” explained Rael. “Something to cut right through the status quo about Apache and MySQL, and possibly even to be stimulating.” Nat showed his assent with an unprintable remark concerning bodily excretions.

Gina laid her hand on my shoulder. “You clearly get the message about our need to expand our reach. Just go out and beat the bushes.”

Two weeks later, therefore, I latched onto a manzanita branch to halt my descent down a slope whose decline made my head whirl, and searched the shimmering forest for a large enough gap to pass through. There were clearly no mobs here, smart or otherwise. Yet it was here between the ferns and the stream that I found the tell-tale footprints, about 20 inches long and with five distinct toes, fresh and firmly planted in the mud.

I slid along gently, covering my clothes, gear, and skin heedlessly with mud until I must have looked exceedingly native myself, till after what seemed hours I was rewarded by the sight of the large, brown, furry back of a monster squatting over the stream. Her hand was suspended in the running water and a sour look on her face indicated that few fish were coming by.

I said nothing, but casually inserted my hand in the water to mimic her. The stream’s frigidity was biting, so I was relieved when the monster jerked her head toward me and withdrew her hand in a huff.

“Excuse me,” I began as casually as I could while rubbing my own distressed appendage. “My name is Andy Oram, I do books and articles about new practices in computers, and I thought you might be able to help with my research.”

“You’re asking for a lot,” replied the creature. “If you wish to learn from a sasquatch, you must utterly relinquish your most cherished habits and live a new way.”

“What is your way of life, Sasquatch? What technology does it call for?”

“My needs are simple,” she said, reaching up to pick a few berries. “The folks down at the junction have whatever I can’t find in the forest, and they leave me alone.”

“So there’s no killer application that could change your life?” I asked tremulously.

The primitive let loose a snort that resounded through the valley and shook the tops of the fir trees. “Son, I don’t let anything change my life,” she said. “My life is this one, right here. I don’t plan on in restricting my view to what I could see over the top of a cubicle wall. You killer folks can tear down all the trees on the ridge to put up your luxury condos and leave me nothing but a patch of the valley under a plastic bubble, but I’ll be right here doing what I do. There are billions of others just like me around the world, too.”

I was seized with a desire to make notes and slid out my laptop, which was the heaviest item weighing down my backpack. She smirked and pointed to it. “That computer is so big and heavy! Look at that stupid, almost useless keyboard. And it takes minutes to boot!”

I protested, claiming that the laptop was considered portable and convenient compared to older types of computers.

She laughed and said it did not matter how small my computer was, but how I interacted with this computer. “It is a big computer because you use it for big things—for applications like word processing and presentations, right?” She affected a nasal tone and said in a mincing sing-song: “I’ve got to get my PowerPoint slides to Rael for the web site!”

This took me by surprise because I had no idea that she knew who Rael was or how I was connected to him. I decided she must have spent time looking around the O’Reilly Network web pages.

She went on. “Your computer runs applications. It takes thinking. It requires the translation of ideas to new media. It leads to headaches, does it not?” She took out a cell phone. “Here one finds services. Small amounts of data that give you exactly what you want. Program them once and leave them alone.”

Now I erupted in exasperation. “But how can you say that you are satisfied with that little cell phone? The input mechanism is cumbersome and the display crippled. How does it let you set up customized access to those services you are so interested in?”

She pointed a leathery, wrinkled index finger at me. “That’s what you’re in business to figure out.”

I felt put on the spot by the sasquatch in a way I had never been challenged before. I had recently heard of something like she was talking about, a little tool called Konfabulator that was becoming popular on Macs and let users do all kinds of tasks through JavaScript, but I did not know what it might lead to. So I tried to return to my first question, even though it had not been to fruitful before, and asked what kinds of applications she would like.

“Two kinds,” she said, “high-bandwidth and low-bandwidth.” I waited expectantly for details.

“The high-bandwidth application is video,” she explained, but not commercial movies. Why are Internet designers planning on distorting your whole network to deliver something that has a lot of perfectly good delivery channels already? My high-bandwidth application would be me and my buddies on video, mixed in with other data.

“The low-bandwidth application is a little harder to describe. It consists of fleeting packets of data about where the berries are thick this summer, and what the mosquito count is on the other side of the hill, and who just entered my territory. It would be so small and delivered so fast that it hardly deserves to be called an application.”

“How would you and your friends define schemas for such requests, Sasquatch?”

“I don’t have the patience to spend two years defining a schema for everything I do! Why should I care about laying out N-ary trees or whether a Form Factor element is a child of the Product Constraints element or the Customer Requirements element?”

“People used to get six-figure incomes for dealing with those issues!” I protested.

“Well, I don’t have a cent to pay them.”

I realized that this might be part of the problem plaguing the computer industry.

“The scenario you offer leaves no entry point for successful businesse models,” I complained petulantly, as if it was her job to provide business models. “It reduces the Internet to a welter of disconnected, atomized data transfers.”

“Not at all! You have not been using the creative powers entrusted to you in order to really listen. Consider now: you don’t expect me to accept data from just anybody, do you? I want to know what forces I’m dealing with. I have damn few relationships in the world, and the ones I add online have to be part of a network of trusted agents.”

She screwed up her protruding, bony face in a look of surprise that I decided was probably feigned. “Didn’t you guys learn anything from the success of the browser and its integration into the desktop?” she asked. “Users have gone beyond individual applications. We want rich interactions mediated by a format of our choosing.”

“Experts would tell you it requires standard ways to conceptualize the organization and presentation of information,” I said.

“That’s also a source of trouble,” she answered. “Just take a fresh look at the Model-View-Controller framework.”

“What’s wrong with it?” I asked. I sensed her disdain right away.

“Don’t you need to go back home and add some checks for exceptions to your XML parser methods?” she said. I was chastened but waited patiently, squatting by the stream. Finally she started up again. “The end-user deserves more than a View. I want to filter and manipulate the data on my end. I need more control than the Controller gives me. That’s just the first issue.

”Then look at distributed databases and see whether MVC fits. Not as well as people think. The C could stand for communication and consolidation as much as controller. All those things are part of searching different sources, choosing the desired results, harmonizing them, and bringing them back.

“And I don’t want to specify when to go out and get the data,” continued the sasquatch. “Do you have servers? Then I want them to serve me. Push it to me when something interesting happens. An RSS-like feed is more suited to me than a SOAP request.”

She flipped open her cell phone and consulted a Geographic Positioning Service device. “If I’m looking for a grove with lots of acorns, I don’t want to query each patch of the forest. I want them to tell me!

“You ask that before I get information, I formulate a request. Do you know what you’re talking about? There may be 20 data stores providing results. And they may all use different schemas. Are you scared yet?” She raised herself to a towering height and shook her mane.

“Any design pattern has to include distributed search. So I doubt an SQL data-extraction methodology will fill the bill. Instead, how about some intelligent collaboration on the server side?

“In short, the data I want is scarcely structured and often irregular. I just want one piece of datum I need right now and I don’t care where you put it. Finally, take this home with you: I care more about my relationship with my correspondent than in the relationships among his data.”

I had to admit that I didn’t understand the path that the sasquatch was showing me and didn’t feel ready to give up my current way of computing. But she seemed firmly tied to her own ways. And as I found my path back through the silent dusk of the forest to civilization, I realized that fortune and glory might lie with the adventurers able to satisfy her needs.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.