Izenda Embedded Analytics and Technology Blog

The x86 instruction set is the life blood of modern IT. While it is easy to criticize for its lack of elegance and archaic nature, the very low end (mobile devices) and the very high end (supercomputers) seem to be the only place x86 is not overwhelmingly dominant. Many other platforms like Spark, MPS and even Intel’s own Itanium are no longer viable options for the overwhelming majority of applications. The platform is very prominent in the very high end and is making inroads into the mobile market currently dominated by ARM. Apple, a long time PowerPC company, recently shifted to x86 and with enormous success.

Is x86 the best architecture? Nope. The fastest or the most power efficient? Not even close. The great x86 advantage is that trillions of dollars have been invested into it in terms of software, hardware, expertise and intellectual capital. A platform shift today would require more investment than the government bailout of Wall Street.

A decade ago the industry did a lot of soul searching in terms of figuring out what’s important in the server world. Was it 64-bit? low power? High IPC?  Lots of threads? Bandwidth? Megahertz? Emulation? In that search new architectures emerged like Itanium and NetBurst (which was x86 based) that just didn’t make it. Companies like Intel, HP and Sun invested billions of dollars creating processor platforms that are not competitive with 64-bit multi-core x86.

Where SQL Came From

SQL was designed for computing needs that emerged more than a quarter century ago. The hard disk became the ideal way to store large amounts of data and has held that position to date. Putting data on spinning platters rather than random access memory created many very complex but ultimately solvable problems. Financial and accounting applications dominated the computing landscape and things like transactional integrity and ACID compliance were crucial. A young guy named Larry figured out how to monetize this and became one of the richest men in the world.

Database Needs Today

The types of problems dominating database science today are very different. Today’s applications like Google, Twitter and Facebook simply cannot function on the standard relational model on any single server that exists today. And while we still have spinning platters, a lot of stuff can now fit in memory. The number of users is now astronomical by 20th century standards. While a large bank may have tens of millions of customers, they will perform very few database operations in a day. Most of these are automated and may only interface with the system a few times a month for very short periods of time.

Contrast this with an application like Facebook. There will soon be hundreds of millions of users, many of which spend several hours a day on the site. What’s worse is that database operations can involve many parties. When you process a payment it’s between your bank and one other party. When you post to Facebook, every one of your 10,000+ “friends” may need to be updated. This is profoundly beyond the scope and scale of the way SQL has traditionally operated.

New Ways of Thinking

While most of these new applications still have SQL components, the bulk of data exchange is done though new techniques that would be considered exotic and even blasphemous by SQL standards. Architectures like SimpleDB, BigTable and Astoria do many things in fundamentally different ways. By sending simple, state-less and sometimes structure-less objects through the web as XML, these platforms solve the basic Create, Retrieve, Update and Delete (CRUD) needs of an application. While this works for day to day updates, a rich enterprise application has needs way beyond this.

The Demand for Strawberry Pop-Tarts

It turns out that demand for strawberry pop-tarts skyrockets right before hurricanes.  Wal-Mart, North America’s largest employer and one of the few companies thriving in the current economy, not only knows this but aggressively does something about it. The real power behind SQL is not so much its ability to store structured data, but how it can transform business and provide insight into customer behavior over time and across many transactions.  While you could say that such analysis is done with cubes engines or BI tools, they are fundamentally based on relational databases.

The Problem with Data Warehouses

The BI industry has long relied on data marts or data warehouses to solve many problems in order to speed up queries and avoid affecting transactional performance. A data warehouse uses an Extract, Transform and Load (ETL) process to effectively make a copy of production data. The data is often reduced, cleaned up and optimized but essentially there is a copy process that usually happens overnight. Social, retail or news events at a much faster pace. A retailer can’t wait until midnight to know what is selling on Black Friday at noon on the east coast.

The Solution is SQL

While the SQL language does not do some things, like forecasting, it provides the foundation of what modern businesses needs to operate in a competitive environment. This is especially true in the increasingly global and highly dynamic times we are in today. SQL has served us well and needs to shed some remnants of an earlier era, but will allow us to leverage the trillions that have been poured into SQL-based technologies. Like x86, it has its rough edges but serves us well for many situations, so much so that all of the advanced features of SQL will need to be supported in future cloud-based systems.