For my guest talk, each year I’ve improvised the presentation, feeling around for what was of interest. This year, I gave a three-part talk:

• A brief history of computing
• Programming on the web
• Computer science as a career

Caitlin Baker from the Analy film class was kind enough to record my talk and post the video:

I really enjoy connecting a bit with the students and sharing my perspective from 40 years in the industry. I have, alas, found it frustratingly difficult to get most of them to engage by asking or responding to questions during the class.

I’ve had the opportunity to get to know two of the students a little better: Luc Street and Izzy Ehnes both joined Webvanta as interns after my talks in previous years. It’s been great to have them be part of our little company.

The post Talk for the Analy High School AP Computer Science Class appeared first on Parting Thoughts.

My focus is on creating something that will live longer than I will — hopefully a lot longer. It turns out that this is not so easy to do!

I have picked 50 years as a goal. Longer would be better; I’d like 250 years! But it is hard to think clearly about even 50 years in the future.

There’s a big stack of issues to consider, from the domain name and hosting to the content management system (CMS) and software maintenance. I’m going to survey most of them in this article, and then dive into a couple of big issues in future articles.

Third-Party Dependencies

With longevity of the site as a key goal, it is important that the website avoid use of third-party services. For example, in current websites JavaScript libraries and font files are often loaded from Google or other content delivery services. Using a third-party server to supply the file has some performance and simplicity advantages, but it makes the site vulnerable to any changes to the third-party service. My top goal is maximum long-term stability, so third-party services are out.

The current site does use some third-party services, such as AddThis for sharing, for ease of development. As long as these are optional services that don’t otherwise affect the site, they are an acceptable risk. When it comes to dealing with social networks, stability and avoidance of third party-services just aren’t possible.

Third-party software is fine, though, as long as it is considered part of the website’s software. For the CMS, I chose to use WordPress, knowing that it is more widely used than any other system on the web and therefore will be around for a long time.

Normally, an unattended WordPress installation is a dangerous thing, considering the number of hack attempts aimed at WordPress sites and the frequent rate of software upgrades.

There’s a whole host of issues here that I’ll address in a future article. For now, let’s put this big messy issue aside, and consider the WordPress software and the PHP/MySQL stack that underlies it as a given.

Offline Files, Online Website

A website does not need to be connected to the Internet; it can exist, in a static form, as a set of bits on any storage device or service.

An offline website is only good as a “deep freeze” backup, however. My guess is that most sites that make it to this state are never seen again.

I am putting together both an offline set of files and a website. The offline files will be my complete data set in raw form. This makes it easy to assemble the content, since no user interface is needed, and it can be as inclusive as I want without a lot of effort.

While I like the idea of having this big pile of bits that will capture a big piece of my life, it is far more important to me to have the most interesting of this content easily available to anyone. And for that, I need a website.

Long-Term Hosting?

To be on the web, the site needs to be hosted by someone. That is simple enough to do, of course, for immediate use; there are hundreds of hosting companies to choose from.

But how do I set up hosting that will run indefinitely? Servers cost money to run and need maintenance and support. Any company that is offering “lifetime” hosting can only be referring to that company’s lifetime. The few I found offering this kind of service did not appear to be big enough that I would bet on their lifetime.

The best approach, putting aside cost for the moment, seems to be to host the site on two different services, with automatic synchronization between the two. This adds cost and complexity that I don’t want to deal with right now, but I’d like to add it in the future.

So it comes down to betting on one company to be the initial host. My guess is that Amazon’s AWS services are likely to be among the last hosting services standing. Their scale, in terms of the number of companies that depend on their services and the ranges of services they provide, as well as their own businesses that use the same infrastructure, is a huge strength.

Amazon’s billing however, is usage-based, so you get a bill at the end of every month and the amount varies depending on traffic. This is not a good match to our “set it and forget it” goal.

Amazon will sell you a “reserved instance” paid up to 3 years in advance, but they don’t offer longer periods than that. The service is about $200/year, when paid 3 years in advance, for what Amazon calls a t2.medium instance. So even if you could pay for 50 years, you’d be looking at a $10,000 bill. (I imagine Amazon might offer a discount for 50 years — or they may not want to commit in any way to that long a period.)

Currently, I am hosting this site at WP Engine, because of their strong support for WordPress security. They might be a great long-term host, but they don’t want prepayment for any longer than one year, and they are too small for me to depend on as a 50-year solution.

One backstop is the Internet Archive’s wayback machine, which caches much of the web periodically. This is a wonderful service, and I am delighted that it is there — but it doesn’t do a comprehensive enough job to be the primary site, or even the primary backup. For example, it doesn’t include all the images and other ancillary files.

So let’s assume we have hosting set up somewhere prepaid for a few years, and that the best we can do for the long term is to have a couple of people lined up to pay the bills, maintain the software, and find the next-generation team when they are ready to retire. Perhaps some sort of escrow account can be set up that will pay the bills in future years.

Finding the Website

If the website is going to have its own domain name, the name needs to be registered at one of the domain registrars. GoDaddy will sell you 10 years prepaid, and Network Solutions offers 25 years. So someone is going to have to pay to renew this, but not for a while.

In addition to domain name registration, someone needs to provide domain name services (DNS). Typically, this is done by the domain registrar, and they do it for free. So if the registration problem is solved, the DNS problem should be solved too.

Another option is to use a subdomain with an existing service — for example, this site could be partingthoughts.wordpress.com, instead of partingthoughts.net.

If you take this approach, however, the site is now tied to a particular hosting provider, which is an unnecessary risk. It might be useful for a backup version of the site, however.

Storing Lots of Content

I have about 100,000 photos that I have taken during my lifetime. Only a tiny fraction of them will make it up onto the web, but they will still add up to a substantial amount of disk space. When I start adding in videos, the amount of space needed will skyrocket.

Fortunately, disk space, and the services that deliver it, have continued to fall in price and increase in capacity. Cost and availability of storage is no longer a limiting factor for this sort of website unless you have vast amounts of video or other big data sets.

In the AWS world, storage is the S3 service. It gives us arbitrary amounts of space for a low price. Some redundancy is built in, and you can pay extra for more redundancy if you want even higher reliability.

Of particular interest for archival storage is Amazon’s Glacier service, which stores data offline. It is significantly less expensive than online storage, making it ideal for backups.

Character, Page, and Image Encoding

It is easy to lose sight of the fact that everything that we store, process, and deliver from our web servers is just ones and zeros — there are no alphabetic characters, no images, no audio.

All of these media types are possible because of some defined encoding of the real-world entity — such as a photo — into bits. For these files to be readable, future software needs to be able to decode them. This is typically done by the browser, so it is dependent on what future browsers support.

Choosing encodings is something we need to do once, and do it right, so that our content will have as long a life as possible.

For characters, ASCII remains the heart of standard encodings. A specific enhancement, called UTF-8, predominates on the web today and provides for encoding of a much larger character set without going to a full 16-bit character. I’m betting that browsers will be able to display UTF-8 text for a very long time to come.

Images are more complex, because it is not just a matter of encoding but also of resolution and compression. I am storing at least two versions of my photos, if they are on the web: a screen resolution version for the web, and the original file for the offline archive.

The file size difference is huge. Images on the website are typically 100 Kbytes to 1 Mbyte, while raw files from my camera are around 25 Mbytes.

For offline storage, where I want the full photo quality to be preserved, I’m saving the Canon raw files. It would be more robust, perhaps, to convert the files to Adobe’s vendor-independent Camera Raw format, but this is a project I suspect I won’t get to.

For the web, I’m betting that JPEG is going to be around for a very, very long time. The number of JPEG images in the universe today is already massive, and it is increasing at an ever-faster rate. Cameras and other capture devices may move on to new formats, but display software — such as browsers — aren’t likely to drop legacy JPEG support for as far in the future as I can glimpse.

One way to reduce the dependency on JPEG would be to use an uncompressed format, but this would dramatically increase the amount of storage required and decrease the performance of the website. I’m going to roll the dice with JPEG.

Initial Conclusions

There does not seem to be any realistic way to create a website that has a high likelihood of persisting unaided for more than a few years. But with a little help from friends, or perhaps from an escrow account, a 50-year goal is achievable with diligent effort.

You can prepay everything for 3 years and it should run unaided, as long as there are no security breaches or truly essential software upgrades. After a few years, however, someone is going to have to start paying the bills for hosting, and once every 10–25 years someone will need to renew the domain name.

If you are intrigued by the idea of creating a personal archive, I encourage you to begin by ignoring most of this article so you can focus on your content. What do you want to publish, and how do you want to present it? Then you can move on to all the issues discussed here.

The post Creating a Lasting Website as a Personal Archive appeared first on Parting Thoughts.

Sure, we’ve been there for years, depending on how you define it. But this is a big leap from anything I’ve had in decades of internet use.

I started with 1200 bit/s modems, had 256 Kbit/s ISDN for years, and have been through generations of DSL and cable broadband, starting with less than 1 Mbit/s and moving up to today’s 34 Mbit/s from Comcast.

We currently have both Comcast (cable) and Sonic (was DSL, now fiber) service. Sonic’s DSL service delivered only about 13 Mbits/s, and we used it only as a backup.

In the selected areas in which they offer it, Sonic now provides gigabit fiber service for $40/month, including a phone line. This is an awesome deal. It is the same price Sonic has been charging for DSL, which is 50 times slower, and even that is a good deal.

In 2011, Sonic ran fiber through several Sebastopol neighborhoods, but it only hooked up houses on one street. My neighborhood was left on Sonic’s “coming someday” list. Five years went by, and I was wondering if I would ever get fiber.

Now it is here! The fiber runs from the pole to my house, into the room that is my office, and terminates optically in the first of three Sonic boxes. None is a modem of any kind! Ones and zeroes just flow on through as either light or as electrical levels.

Sonic put a sign in the driveway while they were doing the work, to complement the two loudly painted Sonic trucks parked across the street.

How Fast Is It?

But is it really gigabit? Not quite, but pretty darn good: more than 700 Mbits/s download, and 100 Mbits/s upload.

That’s about 20 times the download speed of the Comcast connection. If you use the Sonic DSL service as the basis for comparison, instead of Comcast, then the speed increase is more than 50X. Either way, it is a stunning upgrade.

But Does It Matter?

As spectacular as this improvement is on paper, how much difference will it make in the real world? That remains to be seen, but some things are clear.

Independent of the speed gain, fiber should be significantly more reliable than the old phone company copper wires that Sonic leases for DSL service, or the coaxial cable network that Comcast maintains.

In terms of speed, I expect it to make a big difference to me in a few areas.

I keep all of my files synchronized to Amazon’s S3 cloud storage service and/or to Dropbox. The 100 Mbit/s upload speed should make uploads go much faster when I have big files.

Ignoring all overhead, if we look just at raw, peak transfer time, 100 Mbits/s is 12.5 Mbytes/s. If I have 1 Gbyte of photos from a photo shoot, that will take about 1.3 minutes to upload. At 6 Mbits/s (Comcast upload speed), the transfer rate is 0.75 Mbytes/s, so 1 Gbyte takes 22 minutes. (Real-world figures will be lower, but we’re only looking for comparative numbers here.)

Now let’s suppose I have 100 Gbytes of photos and video from a vacation. That will take 2 hours and 13 minutes with the gigabit service, or about 37 hours with Comcast. You could upload 100 Gbytes in an evening using the gigabit service, but with Comcast you’d have to hope you could keep it going for nearly two full days.

Will Video Be Better?

In the past few years, the situation in which I most frequently experience what appears to be a bandwidth problem is when watching streaming video. The image may freeze up, get blocky, or drop to non-HD.

I’ve never been sure who to blame. It could be Netflix’s or Amazon’s servers that are providing the data, or it could be my internet connection, or it could be anything in between. We’ll soon see how much this has changed.

I remember watching a demonstration at an Intel analyst event in the early 90s, at which they showed jerky, postage-stamp-sized video on a PC screen. It did not move me at all; I thought, “who would ever want to watch video so tiny and low quality?”

It has taken almost 20 years of evolution in multiple areas, including faster processors, bigger memory chips, larger disk drives, and a much higher bandwidth internet infrastructure, but those postage-stamp videos are now movie-theater, big-screen quality, and I can stream them straight from my internet connection — a different video stream into every room in the house, if I wanted!

Bandwidth appears to have ceased to be a precious commodity, at least at my house.

The post Gigabit Fiber Has Arrived! appeared first on Parting Thoughts.

I’ve recently assembled a new series of courses in response to the needs that I’ve seen helping dozens of designers build great websites. These courses cover the trio of technologies that make up the foundation of the web: HTML5, CSS3, and JavaScript/jQuery. In addition, there’s a course on mobile design, which has become increasingly important.

If you’re interested in developing your skills in these areas, I think you’ll find these courses to be a very productive use of your time. Follow the links below for details.

• Online courses for web designers
• HTML5/CSS3 online course
• JavaScript and jQuery online course
• Mobile web design online course

The post New Online Courses for Web Designers: HTML5, CSS3, jQuery, and Mobile appeared first on Parting Thoughts.

Is it an entirely new category of device, as some have claimed? It’s true, of course, that it’s not much more than iPod Touch Jumbo. From a technical perspective, it’s cool, but not earthshaking.

From the perspective of how it will be used, however, I think it will create a new category. It is the most intimate merger yet of computing and consumer electronics.

A Clear View, Not a Short Distance

A dozen years ago, I wrote an editorial for Microprocessor Report about tablets being the future of news, and suggested that they would become common living-room appliances. As futurist Paul Saffo is fond of saying, never mistake a clear view for a short distance. My vision was clear, but the distance was a decade more than I anticipated.

My previous startup, PhotoTablet, was founded to build a tablet that would be a consumer’s digital photo world. For a variety of reasons, it wasn’t realistic, and PhotoTablet turned into Fotiva, a software company.

So it was with some interest that I’ve watched the emergence of the iPad. As usual with any major Apple introduction, it was way overhyped by some of the press, and it’s been attacked by many in ways that just miss the point.

An iPod Touch Jumbo is enough for me, and I expect it will be quite successful. I think this will be an excellent living-room device. It feels (or, I should say, it looks like it will feel) much friendlier than a notebook computer for reading the news while drinking my morning coffee, or to share photos while sitting on the couch, or to casually look something up on the web while away from my computer.

I expect it will also give eBook readers a run for their money. Will users choose a dedicated device without color or video but with a lower price, longer battery life, and possibly an easier-on-the-eyes display, over one with many uses and color video? My gut is that most will not, and that over time Apple will match or exceed the available libraries for the dedicated devices.

Not Without Its Limitations

It does have a number of limitations, of course, but as the first member of a family it looks pretty darn good.

The limitation that frustrates me the most is the lack of Flash support, which makes Apple’s claim that this is the best Web experience ever just wrong. Apple could surely put Flash on it if it wanted to. The lack of Flash support must, I believe, indicate some sort of pissing contest between the two companies. There is no shortage of bad blood between Apple and Adobe, and there is every appearance that Apple would like to see Flash die, though I don’t understand their motivations.

I don’t see this as a business device, or as something I would take traveling. If I’m going to take something this big, I’d take my MacBook. It is not just the lack of a keyboard and physical protection, but the lack of multitasking and rich applications. I would take it over to a friend’s house to share pictures, though.

The Long Road

No company could have built a device like this from scratch. Apple was able to do it by cleverly extending its product range over an extended period.

The iPod started out as nothing more than a slightly better MP3 player, which was catapulted to dominance by its iTunes integration and great marketing.

Then Apple added the ability to play tiny videos.

The big leap was merging with a phone and adding applications, which initially were proprietary Apple apps only. Apple would have had a very hard time jumping into the phone business if it were not for the iPod, which gave lots of people a strong reason to buy an iPhone independent of its quality as a phone.

Then Apple opened up the App Store, and the iPhone became the most important new computing platform since the Mac and Windows.

And now Apple is leveraging all of that design evolution, and the momentum of the iPhone, to create the first really exciting consumer tablet.

It’s the kind of strategy that few companies have ever been able to execute on so well.

The post The iPad: Imperfect, But Enough for Me! appeared first on Parting Thoughts.

The slides are available at slideshare, but they’re sort of dull without the talk track.

The post A Brief History of the Web, and a Glimpse of Its Future appeared first on Parting Thoughts.

I’ve been on LinkedIn for many years, and found it moderately useful as a sort of distributed address book and business contact list. But its role in my life was very small, and entirely limited to business contacts.

Three years ago, I started getting invitations from friends on Facebook, which I accepted but rarely did anything with it.

About a year ago, the pace of invitations I was getting climbed, and I put some work into my profile. I connected with a bunch of high-school friends I haven’t seen in 35 years. It has been fun, but odd, to get little glimpses into the life of these folks—some of them were a big part of my growing up. The odd part is that, for the most part, these are people I never would have had any contact with if it were not for an online social network, and it’s unlikely I’ll ever see them in person.

A few months ago, I started paying more attention to Facebook, motivated more by business than personal desires. I went through my address book and invited people to connect; and I scanned the contacts of my contact to find other people I knew. In the past 20 years, I’ve worked with hundreds of people, and Facebook has proven to be an easy way to reconnect, however tenuously, to many people who I would like to stay connected with but realistically I wasn’t likely to, out in the “real world”.

With a few days of effort, I increased my Facebook friends from 25 to 250. (If only I had started 10 years ago, I would have thousands of “friends”.)

And then, of course, there’s Twitter. I’ve had an account for a couple of years, but only used it occasionally, usually at conferences. Recently I’ve gotten more involved, though I’m still not a prolific twitterer. (More on Twitter in an upcoming past.)

I’m of the wrong generation for all of this to come naturally, despite being something of an early adopter. I’m just starting to get real value, both personal and business, out of social networks, and I still have more questions than answers. Among them:

• Do I maintain separate networks of personal friends and business friends?
• Can I get myself to post frequent thoughts and updates?
• Do I use Twitter, Facebook status, and LinkedIn updates in concert, or do different things with each?
• How do I filter out the stuff I really care about?

Separating Business and Personal Networks

One thing I’m still coming to grips with is the blending of my personal and business contacts. LinkedIn is almost all business, but Facebook and Twitter are a mix (especially Facebook).

The easiest thing to do was to give up on separating my personal and business networks. That’s what I’ve done so far, though I doubt that it’s the best strategy.

The question is, do both groups of people have the same interests? For the most part, I think not. There’s also an interesting twist added by mixing the two groups up, but ultimately I think I’ll separate out my family and close friends network, in one way or another.

Dealing with the Flood

One I had hundreds of contacts, and the social network news feeds became prominent, the next challenge has been dealing with the flood of status updates, tweets, and so forth. To read all of it, even without responding to anything, seems like it would take hours every day.

My solution, so far, has been to simply dip in an out when the mood strikes me. I’m sure I’m missing all sorts of interesting stuff, but I haven’t wanted to devote the time to sorting through all the cruft to get to the nuggets. Some kind of filtering seems like the only real solution.

The Instinct to Communicate

It takes a lot of time and energy to make effective (or even ineffective!) use of all these social networks, and in the early days of using them the energy in is a lot higher than the value coming out. I have no doubt about the long-term value, but it’s still hard to spend the time it takes to keep up with even one of my online social networks.

I’ve been blogging for three years and have never sustained anything like the frequency of posting that I aspire to. I have all sorts of articles running around in my head—the legacy, perhaps of a dozen years writing newsletters and magazine columns. But they take time to commit to bits. What you’re reading now is one that’s been rolling around inside for a couple of months and finally made it out, sitting in a cafe in Santa Rosa while I wait to pick up my daughter from ice skating.

I’d like to get up to at least one blog post a week. And to several tweets a day. It takes a big shift in mindset, though, to communicate this frequently and proactively—it’s an opportunity that didn’t exit prior to online social networks. We’ll see if I get there.

My Success Story

The biggest chunk of business value that I’ve ever gotten from these networks occurred a few weeks ago, when I was looking for a couple dozen web designers who I could interview and get feedback on our new product. I’d exhausted my direct network, and the step to cold calling is a big one.

So I used a LinkedIn question, in addition to a Facebook status update and a Tweet, and with a few days I had introductions to a few dozen people. Because these were introductions, and not cold calls, even though I had no prior direct contact with these people more than half of them have spent an hour with me in a web-based demo and phone discussion.

These networks were a fantastic resource for me. They allowed me to make a casual request of hundreds of people to whom I would have hesitated to email, and I got responses from a good scattering of them—and often not from the people I would have predicted, if I had to select a smaller number to contact.

The post Adventures in Social Media appeared first on Parting Thoughts.

A month or so ago, my wife’s PC’s hard drive starting acting up. I replaced the drive and decided to take this opportunity to “upgrade” the machine from Vista to Windows XP. For some reason, the XP install decided that the machine’s built-in memory card readers were drives C, D, E, F, and G, and the hard drive was H. By the time I noticed this, I had reinstalled a bunch of apps, and although it is an odd configuration, everything worked fine. I did a brief google search looking for an easy way to change the drive letters, and it appears that there’s no easy solution for changing the letter assigned to the boot drive. So I decided to leave it.

The only problem I’ve had is with the drivers for the HP all-in-one printer. HP’s installer appears to be hardwired to look for drive C, and the install fails. I tried various tricks, even putting in a memory card into “drive C” and trying to install from there, but nothing worked. After hours of frustration, I decided to call HP service. That was the beginning of the real frustration.

Turning “service” into sales

After navigating through a circuitous voice menu system and waiting on hold, I finally got to a live person. After providing my serial number, they told me that they would be glad to help me, but that it would cost $35 since the printer was out of warranty. I asked if I would have to pay the fee even if they couldn’t resolve my problem, and the “support” (really sales) rep assured me that they would connect me with an expert who would work with me until the problem was resolved. With some trepidation, I agreed to the charge.

They then transferred me to a woman who seemed to be starting over. She had no idea what my problem was, or that I had paid for the service call. After another 20 minutes, we got this worked out, and started in on the actual problem.

She then launched into a sales pitch for how for “just a few dollars more” than the $35 service call fee they would exchange my printer for a new model and refund the service call fee. I asked how many dollars. She ignored my question and kept on with a pitch for the new printer. After asking literally five times for the exact price, she finally gave it to me: $250. That’s more than a few dollars above $35 in my book, and the printer works just fine with the other computers on which the driver software will install.

I told her I didn’t want to do the exchange, but wanted to solve the problem. After a very brief discussion, she concluded that my printer was just too old, and wouldn’t work with the “new” drivers, and that was my problem. This is total BS—the issue had nothing to do with compatibility between the drivers and the printer, and the driver software on their site was dated 2007, just a year newer than the printer. The problem is their software’s brain-dead installer can’t deal with the hard drive being something other than C. Not only could she not propose a workaround, she couldn’t seem to even understand the problem. Some expert.

She launched back into the sales pitch for selling me a new printer. At this point I said I wanted to speak to a supervisor. She tried again to get me to buy a new printer. I said I wanted to talk to a supervisor. She said none was available then, but one would call me back tomorrow.

“We’ll call you back”—never

I waited three days, and no one called. Finally I called back, went through another 10 minutes of voice menu hell, and explained the situation once again to a live person, who once again was no help. I asked for a refund on the service call fee, since they weren’t able to help me, and she said she had no way to do that. I asked again to talk to a supervisor, and she said one would call back.

No one ever did. After a few more days, I contested the charge with the credit card company and bought an Epson printer to use with that computer. It just wasn’t worth any more time or grief trying to get that printer working with that computer.

HP’s service operation is broken. They don’t understand what they’re talking about. They don’t seem to care about solving your problem. They push expensive upgrades on you even after you’ve paid for a service call. And the policy for responding to requests to speak to a supervisor seems to be to say one will call you back, but no one ever does.

My first job out of college was working at HP. I used to have a great respect for the company. I’ve been loyal to their printers for 20 years. And because of their ineffective and irresponsible service operation, I am unlikely ever to buy another one.

Update—HP Follow-Up

After a couple weeks, I started getting phone calls from HP folks in the U.S. who were apologetic and eager to help. I believe these calls originated from a complaint email I sent in, but maybe this blog post had something to do with it too.

By this point, I had replaced the printer, and all I wanted was a refund of the service call fee, which they gladly provided. In fact, they called back repeatedly when I was slow to gather the information I needed to give them. These people were typical of the HP of old (albeit engaged in damage control). If I could have reached them when I asked to speak to a supervisor the first two times, all of these problems would have been avoided.

One of the HP people I spoke with acknowledged that my problem with “someone will call you back” and then no one ever does has been an issue with this call center.

So my conclusion: HP has a big problem, not at the core, but with the way they have offshored their support. The offshore support organization does not reflect the values or quality of service for which HP has been known for decades. Maybe a better-managed offshore operation could solve these issues. Or maybe the savings from sending support offshore are offset by less tangible, but very real, costs. One cost: I remain disinclined to buy an HP product in the future, because I don’t want to have to deal with this support organization.

The post HP’s Service Disaster appeared first on Parting Thoughts.

Since I’ve been working with Ruby on Rails, I’ve been thinking about how it could be made easier for people to learn. I believe there are vast numbers of web designers and developers who would find Rails a very useful tool, and who could improve their productivity—and their satisfaction with what they’re producing and the process of producing it.

One thing led to another, and on February 20 and 21, my colleague Christopher Haupt and I will be presenting our first Ruby on Rails QuickStart seminar in San Francisco.

We’ve designed the seminar for web designers and developers with only minimal programming experience. We’re providing a pre-built site, which we’ll walk through during the seminar, that attendees can use as the basis of their own sites. We’re also very close to a deal with a hosting provider to offer free hosting for a month, so we can help attendees get their sites deployed before the seminar is over. We’ll provide each attendee with the NetBeans IDE, deployment scripts, and everything else they need to immediately build and deploy Ruby on Rails web sites.

I’m really looking forward to the seminar and hope some of my readers can join me there.

The post Ruby on Rails QuickStart Seminar Launched appeared first on Parting Thoughts.

There’s all sorts of questions being raised about the speed of response. In time, I suspect we’ll learn a lot. For now, speculation on just how quickly the response was mustered, whether it could have been done more quickly, and what caused any delays is just that—speculation.

Even more puzzling is how this could happen to begin with. There’s a huge opening between the bridge towers—this is not a tight fit, even for a huge cargo ship.

It just so happens that one of my sites, BoatingSF.com, tracks ship movements on the bay. Anyone who looked at my real-time ship tracking page within an hour of the accident could have seen an instant replay of the ship’s track. And it should have been reasonably simple for me to access the historical data.

My perfect storm of server data ugliness

This data comes from two AIS receivers I operate, which receive VHF signals that all commercial ships are required to send that encode their position, speed, destination, name, and dimensions. These reports, which arrive at the rate of a few per minute, are streamed up to my server, where I decode them with some custom PHP code (this predates my involvement with Ruby and Rails) and stuff them into a database. Every five minutes, a cron job extracts a summary from the database and generates an XML file. The web page has a Flash movie that reads this XML file to control the animation.

Unfortunately, for performance reasons, I don’t keep decoded position information that’s more than one hour old. I’m going to rearchitect the solution to make this possible, but when I designed this almost two years ago, it was all I could do to get it working, and other projects got in the way of further optimization.

I do archive the raw AIS data stream, so I can go back and process it later to get at historical data. Several private and government agencies have used this data for various kinds of analysis projects. Until recently, I stored this raw stream in the database.

A couple weeks ago, something went wrong, and my simplistic scheme began to torment me. My simple database configuration on an old VPS account didn’t deal well with tens of millions of records.

I had the system set up to send me an email when a database failure occurred—and I started getting 100,000 such emails a day!

There’s another article to be written here, but suffice it to say that you shouldn’t do this (and I don’t any more)—write the errors to a log file, use logrotate or somesuch to keep the files from getting too big, and then use something like logwatch to warn you when the logs have errors in them.

The fix creates a new problem

In my hurry to stop the mail deluge, I changed the code to put the raw AIS stream into a log file instead of into the database. And in the rush, I forgot that the raw AIS data lacks a built-in time stamp. So when I went to dig out the data that would show the Cosco Busan accident, I found that I had no timestamps for any of the position reports! This meant I couldn’t just look for 8:30 am Wednesday but had to analyze ship movements to find the accident.

I also had to write new code to pull the reports from the log file, decode them, and stuff them into the database for further analysis. And I wanted a different zoom region for the display, which took additional work.

About 12 hours later, I had an animation of the accident completed. It doesn’t show actual time, since I didn’t have any timestamps to work with, but it animates on the assumption that the pace of ship reports is roughly constant (which it should be).

I had expected that the ship would have been heading for the space between the towers, and veered a bit off course. The reality is that the ship was going nearly parallel to the bridge, until it turned sharply and headed straight for the center tower! And there was a tug following closely behind. Unless there was a catastrophic steering failure, which seems unlikely given that the ship continued on under apparently good control, there’s some people with a lot of explaining to do.

Next up: a new architecture

This debacle (my server’s, not the ship’s) has spurred me to begin thinking about a new architecture for the system. I want to be able to pull any past window of time, and zoom in on any region, without custom programming.

Perhaps I’ll move it over to Ruby while I’m at it. One of the reasons it too me so long to create the Cosco Busan animation is that it had been almost a year since I had touched the PHP code, and it is an ugly thing! It is hard to remember to put semicolons at the end of every line, and empty parentheses after function calls that take no arguments, and so forth, now that I’m accustomed to a language that doesn’t have such requirements.

Making it even more complex is the convoluted Flash code that creates the animation, which requires dealing in yet another language and the vagaries of the Flash timeline interface. I’m not sure I see a way out of having the Flash code, but at least I can get rid of the PHP.

The post Tracking the Cosco Busan appeared first on Parting Thoughts.