Skip to content

25 Years of OLAP – A personal view

February 9, 2012

In the beginning there were spread-sheets.

VisiCalc was the first spread-sheet program available for personal computers. It probably was the application that turned the microcomputer from a hobby for computer enthusiasts into a serious business tool. VisiCalc sold over 700,000 copies in six years.

Soon more powerful clones of VisiCalc were released. Eventually, Microsoft Excel (introduced for the Macintosh in 1985 and for Windows 2.0 in 1987) became the dominant spread-sheet tool, which in recent years has become the most important end-user tool (via Pivot Tables) for Microsoft’s OLAP server software.

I have used 5 OLAP software since the early 1980s. Here is a very brief précis of each of them:-

Wizard –> System W –> One-Up/Prism

In the early 1980s, Wizard was initially developed for use on Comshare’s time-sharing mainframe computers. Wizard extended 2-d spread-sheets into multi-dimensional databases, which were mainly used for management reporting, budgeting and financial modelling. There were 2 PC versions of it: One-Up and then Prism but both were sold, primarily, with the mainframe versions.  

So, what is meant by “multi-dimensionality”?

Consider: You want to monitor several Sales measures (volume, price, sales) over Time (say, 12 months) and you also want to store and analyse this data by Customer, by Product and by Location. In short, 5 “dimensions” in all. How do you visualize this amount of data?

My analogy is based on a “Filing Cabinet” …

Start by taking a single spread-sheet (Measures (rows) by Time Periods (columns)) – You could treat this as one piece of paper (for 1 Customer, 1 Product, 1 Location).

Next, create and place the above data for all your Customers in one drawer of the Filing Cabinet.

Next, place all your Customer x Product data in one physical Filing Cabinet.

Next, place all your Location x Customer x Product data in separate physical Filing Cabinets (in one room).

Aggregations (summations) of the above data would require additional Filing Cabinets.

Now, if you add another dimension – you will need lots of Rooms. Right?!

And so … with lots more Dimensions (which is not uncommon these days) … Rooms become Floors become Buildings become Towns become Countries become Planets etc.

Hopefully, the above analogy highlights (at least) 2 problems associated with multi-dimensionality:-

There is a high degree of Sparsity, that is, not every Customer buys (or sells) every Product in every Location; and

You would have to travel to far-flung places to actually get hold of your required spread-sheet!

Moving on, Wizard’s success grew and it needed to be marketed world-wide. The name Wizard could not be used in north America and so it was re-branded as “System W” (because of its W> prompt on the time-sharing computers).

By the early 1990s, System W was used by the top 3,000 companies world-wide and could therefore claim to be (at least in my opinion) the “Mother” (literally) of all DSS / OLAP application software.

Essbase

Essbase means “Extended Spreadsheet Database”. It was first released by Arbor software in 1992. Why was Essbase developed? To fill a gap in the PC client/server (i.e. not mainframe) market. No OLAP database existed in 1992 which offered true client/server processing with concurrent read/write access as well as the capacity to handle large volumes of data (100mb, not GB!!!).   

Essbase grew to replace System W for traditional management reporting and budgeting applications.

It was a robust technology with strong modelling rules, eventually capable of handling gigabytes of data, with a tried-and-tested Excel Add-In, the latter being a real differentiator.

Buttt … Essbase was primarily a numerical database (it did not handle textual Measures initially), without any data integration (aka “ETL”) component and no GUI as such. It was also expensive. Taken together, this was a gap waiting to be filled by others.

That said, Essbase could, arguably, be considered to be one of the 10 most influential technology innovations of the 1990s (along with Netscape, Google etc.).

Gentia

Gentia means “For Everyone” – the first version was released in 1993 by Planning Sciences. The functionality concentrated on large scale EIS/OLAP applications and contained some features which were a vast improvement on the competition. In particular, it offered a superb (for the time) development (Book/Page) environment and seamless integration between the screens (“Pages”) and the OLAP database (“GADB” and later “GentiaDB”).

Buttt … Gentia could not do it all! For example, it had weak modelling rules, was not suitable for budgeting applications, did not possess a data integration tool, a complex programming language (“GDL”) and, worst of all, was very expensive!

Arguably, Gentia should have been acquired by a larger organisation which would have taken it to the next level. It was not and, sadly, its demise was as swift as its rise.

Analysis Services

In October 1996, Microsoft purchased OLAP technology from Panorama and also hired some of the key relevant personnel from the same company. Initially codenamed “Plato”, it was shipped with SQL Server 7 and eventually “integrated” with SQL Server 2000. Arguably, it was not until SQL Server 2005 (SP1) that Analysis Services became serious production-worthy software.

The coming of Analysis Services spelt the “death-knell” for the independent EIS/OLAP vendors, such as Gentia, because (a) it was (and still is) relatively cheap, (b) not proprietary (i.e. it was used with SQL databases), and (c) it was from Microsoft, which was a large, stable and growing organisation at the time.

From my perspective (as a Developer), the Analysis Services databases (“cubes”) were much smaller than, for example, the Gentia ones which meant that much larger applications (more Dimensions, more Measures) could be developed. With the 2005 version came the Partitioning functionality, which is an effective way to overcome the aforementioned Sparsity problem.

Analysis Services continues to evolve, with the recent release of the “PowerPivot” functionality and the up-coming “Tabular” option, both of which use column-based (“Vertipaq”) technology.

WhiteLight

WhiteLight Systems started in 1992 with a product on NeXTStep but did not get going until Michael Brill of Sybase heard about it in 1994. Sybase then decided to fund its development until 1997 after which it was funded by private equity firms. Its sales peaked around the turn of the century but it was very expensive software which was sold mainly to banks and after 9/11 direct sales of it plummeted.

From my perspective (as a Developer), the key feature of the software was that it did not store the fact data.  Just imagine that for a moment! When you initiated a query then WhiteLight converted the query into SQL, fired the SQL at the relational database, retrieved the results, performed any aggregations / re-calculations in-flight, cached the results  and presented the results to you. Quickly. It was “like running Excel against a Server”.

Finally …

I am also a Developer.

I spend most of my time delivering information systems – MIS / DSS / EIS / DW / BI.

Over the past 25 years (and over 50 projects), I have had to wrestle with integrating completely different technologies from different (or even the same) vendors on the same project, all of which was oblivious to the end-user.

Arguably, not since Comshare’s “Commander” offering of the late 1980’s has a single vendor provided an “integrated” suite of components for an information system:  relational database, data integrator, olap database and reporting functionality.

Arguably, that offering today is SQL Server 2008 R2 … with Windows Server 2008 R2 … they are best together. 

Advertisements

From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: