Catching The Next Wave

Note: The Pages o’ Peat have moved to http://peat.org/ — please update your bookmarks and references accordingly. Thank you!

Every now and then a set of technologies gets twisted together by a small group of dedicated people, and a new industry is born — a watershed event that demonstrates a new way of thinking about things, and throws out a lot of old rules.

There are a three that are coming together to trigger another watershed.

The first is open, popular, mobile Internet devices.  Think Blackberry, iPhone, or the slew of new MIDs that Intel showed off a few days ago in Shanghai.   These are built around the assumption of ubiquitous access to the Internet, high resolution displays, multimedia capabilities, and a bit of horsepower under the hood.  Any college student can get their hands on the Android or iPhone or Windows Mobile SDKs and build a hot little application in their spare time.

The second is web services.  It doesn’t matter if it’s WS* or REST or XML or JSON — the point is being able to query and manipulate data at a distance, with open protocols across public and private networks.  Pick your web framework of choice … building a web service is almost a drag and drop process today.

The third and final piece is cheap and scalable cloud computing.  The physical infrastructure capable of serving billions of transactions is available to anyone with a credit card and a little spare time on the weekend.  Amazon’s Web Services, Google’s App Engine, and a slew of smaller providers sell scalable computing and bandwidth by the hour and gigabyte.

These three fit together to form a fundamentally different picture of mobile computing:  light weight applications that fit in your pocket that take advantage of the local hardware, but seamlessly tap into “Internet scale” computing power and storage.

I’ve talked with a dozen entrepreneurs in as many months who are exploring these waters.  Streaming media (push and pull), information discovery and analysis, mobile social interactions, and location aware applications all depend on this trinity of capabilities.  I’m just one guy in a groundswell of people who are looking at the landscape and thinking “hot damn!”

What makes this so exciting is how easy it is to do today.  You don’t need a dozen engineers and a multi-million dollar budget.  You don’t need to negotiate with a corporate gatekeeper.  You don’t need to pitch to VCs.  You don’t need to wait.

2009 is going to bring a wave of media rich, location aware, always connected mobile applications to hundreds of millions of people.  I’m confident we’ll see a real forehead slapper by the end of 2008 — a tool or service that is painfully obvious, but fundamentally changes how we think about a day to day task.  It’ll make a millionaire or two, at the very least.

This will be fun.  :)

Co-op

I attended an interesting presentation today at the OTBC: “Co-Ops, What Are They, and How Do You Start One?”

The gist of the co-op idea is that for certain groups of people or companies, coming together to share resources can be far more efficient than simply going at it alone. A good example that everyone’s heard of is OceanSpray, the cranberry company. It’s a group of independent cranberry farmers who came together and formed a corporation for distributing and marketing cranberry products. It’s run democratically, profits are shared amongst it’s members, and it’s very successful in it’s own right.

Adam DuVander (from the Portland Web Innovators) and I talked afterwards about how this concept applies to something closer to home: professional software development. There are lots of independent software developers and consultants in the Portland area, and while the technologies and applications vary wildly, there are probably some very common frustrations that could be aided under a co-op structure.

Ask any independent software developer or consultant what their biggest gripes are about running a business, and I’m pretty sure the same three things will show up on everyone’s list: managing money, legal issues, and insurance. Invoicing, book keeping, solid contracts, good NDAs, taxes, health and vision and dental plans and all those sorts of things are necessary for running a successful business … and have nothing to do with delivering high quality software.

My knee jerk thought is that a co-op could go a long way toward solving these problems. Larger groups have more bargaining power for getting accounting, legal, and insurance services … and centralized bookkeeping and payroll would make a lot of people’s lives easier: wouldn’t it be nice to do business as an independent software developer, to get paid for your work, and to not worry about the rest?

Anyhow. Does this sound like a compelling idea to anyone else?

Update:  You’re welcome to join the discussion on the Portland Web Innovators forums.

Looking for a Sr. Web Developer

With RailsConf in town this week, it’s a good time to mention that Blue Hill Solutions is looking for a Sr. Web Developer.

Specifically, we’re looking for an organized Rails guru who enjoys coaching others, and can step up to a leadership role in our development group.   We want a well rounded individual with at least ten years of software development behind them — bonus points for experience with e-commerce, scalable architecture, interface design, testing, and application deployment.

This is a salaried position, with aggressive bonuses, perks, and full health + dental coverage.  Our office is located in a trendy neighborhood in north Portland, with easy access to downtown, great accessibility via public transit, and within walking distance of a variety of good restaurants and coffee shops.  Although we don’t pay for relocation costs, we’re more than happy to talk with developers from other parts of the country!

Blue Hill Solutions is a start-up with a variety of web-based products, including intranet communication tools and  an evolving e-commerce platform.

Sound appealing?  Send an e-mail to hiring@bluehillsolutions.com with your resume and a description about why you think you might be a good fit.

A Ray of Hope

Corporate environments kind of freak me out. The stereotypes are awful — working in a company with thousands of other people is a soul sucking, beige on beige, cubical farmland of mediocrity.

Coming to the Corporate Communicators Conference has changed my opinion a little bit. There’s almost 500 people here, representing some of the biggest corporations in the United States, and it’s pretty much a non-stop rally for making work environments more open, more communicative, and more human. At the moment, there’s an executive on stage from one of the biggest insurance companies in the world, talking about how uncomfortable it can be to open up on a weekly blog, but how important it is to be genuine, and to respond honestly to critical feedback.

I don’t expect the universe to suddenly reorient itself, but it’s good to see some cracks in the “nameless, faceless” corporate facade.

Try Panicking

Found in the OmniPlan manual:

Heh.

Tax Filing

Just a quick recommendation — if you need to file 1099s for your contractors, I recommend eFileForBusiness.com. It ain’t web 2.0, but it’s a piece of cake to use, and it’s reasonably priced.

Best Work Environment, Ever.

You know you’re with a good company when your boss sends an e-mail like this.

Rails + Joyent Accelerator Benchmark

Well, this is fun. Here’s the configuration:

• Joyent Accelerator 64-L — We stripped it down and built it back up with the bare minimum Blastwave packages we need to run our apps (surprisingly few).
• Rails 1.2.2 Application — Moderate DB complexity, loaded with fixture data, and using acts_as_ferret extensively. No optimization at all (caching, query tweaking, etc.).
• Mongrel 1.0.1 — Running in production mode as a cluster with 4 instances (I think the server has 4 processor cores).
• nginx 0.5.14 — Built from scratch, and running the generic Rails/Mongrel configuration found on the wiki.
• PostgreSQL 8.1.4 — Out of the box configuration from the Blastwave package (nothing special, no ANALYZE statements).

We’re seeing about 50 connections served per second, with 80% of pages delivered within 150ms, and zero dropped or failed connections. I’m pretty sure we can double that if we spent any time optimizing the application and server stack.

Not bad .. but the big fat caveat is that it’s 11:00 PM so our box obviously isn’t loaded very heavily. We can burst up to 95% CPU utilization on the machine, so of course these numbers don’t reflect reality of peak hours. That said, we’ve never seen load reported over 0.2, even during peak.

I’m happy!

Update:  I should note that the 50/sec rate is for a pretty dynamic application with some bells and whistles; connections for static content are in the hundreds per second, but not too applicable for our purposes.

nginx on Solaris

We’re continuing our evaluation of TextDrive / Joyent Accelerators, and things are going well. Tech support is responsive to my odd requests, the forums are lively, and the system itself is quite speedy.

Tonight we’re installing nginx as the load balancing proxy for our Rails applications. It’s been personally recommended to us several times, battle tested by a few independent folks who are into these sorts of things, and received a lot of good press in the Rails community — and it’s mysteriously Russian. So, we’re quite interested to take it for a spin on our Accelerator.

First, there isn’t a Blastwave package for nginx, so we built from the tarball (which requires the PCRE sources). It’s a pretty simple, although it took some troubleshooting. The snag was that the PCRE build used tools that weren’t in my default $PATH, but were in /usr/ccs/bin.

So:

$ export PATH=$PATH:/usr/ccs/bin


… and then we’re off to the races:

$ wget http://sysoev.ru/nginx/nginx-x.x.x.tar.gz
$ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-x.x.tar.gz
$ gtar zxf nginx-x.x.x.tar.gz pcre-x.x.tar.gz
$ cd nginx-x.x.x
$ ./configure --prefix=/opt/nginx --with-pcre=../pcre-x.x
$ make
$ sudo make install


PS: Rails 1.2.3 was released today. And, happy birthday, Mom!

Joyent Accelerators

We’re test driving a Joyent Accelerator to see if a small cluster of them could be a reasonable replacement for our current hosting infrastructure. So far, so good — very speedy, and it looks like it’ll be easier to scale when we need more horsepower. For those not familiar, Joyent’s Accelerators are Solaris containers, Sun’s OS-level virtualization technology.

There are only two snags I’ve hit so far:

Clutter. When you get an Accelerator it comes preloaded with webmin, apache, php, mysql, courier, and all sorts of other things that people might want. However, we’re just deploying Rails apps, so the cruft has to go. We’ve spent a fair amount of time getting rid of the clutter … it would be GREAT if we could simply buy a bare-bones Accelerator.

Compilation. The Accelerators are well equipped with Blastwave packages for GCC and related tools, but Ruby wants to use Sun’s compilers for building native extensions — pretty critical to things like Mongrel or acts_as_ferret. There is a posted workaround here, but it could be a pain to diagnose if you’re not familiar with how gems are built.

Otherwise, it’s all rock and roll. Kudos to Joyent for getting this service off the ground — despite it’s rough edges, it’s a great service.