• Historical notes on analytics — the pre-computer era
• Historical notes on analytic terminology (this post)
  
• Historical notes on analytics — departmental adoption

Discussions of the history of analytic technology are complicated by the broad variety of product category names that have been used over the decades. So let me collect here in one place some notes on how (and when) various terms have been used, specifically:

• Management information systems
• Decision support (systems)
• Report writer
• Fourth-generation language
• Executive information system
• Business intelligence
• OLAP (OnLine Analytic Processing)

Obviously, I can’t cover everything in this post. Omissions include but are not limited to:

• Anything in the data warehouse/data mart area. (For one thing, I don’t want to deal with the whole Inmon/Kimball dispute.)
• Anything in the predictive analytics area (but see the first point in a 2009 SPSS post).
• Terms I’ve recently sponsored, such as investigative analytics or machine-generated data.
• Big data (analytics) — I just discussed that mess a week ago.

The first prevalent term I recall for “information technology” was management information systems (MIS). I mention that mainly to note that it actually sounded a bit analytics-oriented, and hence to point out that in the old days — 1960s and so on — it didn’t seem necessary to name a separate category that amounted to “analytics”.

Meanwhile, the first prevalent term I recall that covered much of what we’d now call “analytics” was decision support, or decision support systems (DSS). I think DSS was always ill-defined, with multiple subcategories, just as analytics is today. The heyday of this term was in the 1970s/1980s.

Report writers were around in various forms for decades; consider for example the early 1970s history of Cullinane/Cullinet. By the time I became an analyst in the early 1980s, these were mainframe tools that let you specify paper reports, and the market leader were probably Pansophic’s EASYTRIEVE and Informatics’ Mark IV. According to marketing, they could be used by non-programmers; in reality, they were a much easier way for programmers to do what end users asked. They were used both for one-shot queries and, as their main design point, repetitive reporting.

The report writer category then survived into the era of business intelligence (BI). (Indeed, Cognos’ big integrated BI tool early this century was called ReportNet.) More on that in the BI discussion below.

The term fourth-generation language (4GL) was widely used from the 1970s through the first part of the 1990s. Usually, a 4GL was:

• A vendor-specific programming language …
• … sold in connection with an interpreter …
• … that was particularly good at database manipulation.

In particular, the original 4GLs:

• Were primarily sold and used for analytics.
• Were often sold/used on a remote/timeshared basis.
• Often had some kind of (pre-relational) DBMS bundled in.

Classic examples of such 4GLs included FOCUS (the core product of Information Builders), RAMIS, and NOMAD; SAS arguably started out as a product of that kind too. Starting in the 1980s, however, 4GLs were used more generally, and indeed survived as an OLTP (OnLine Transaction Processing) technology long after they were supplanted by BI tools for most analytic purposes.

The last pre-BI term I want to mention is executive information system (EIS). EIS was in essence the 1980s term for “dashboard”, although the technology was much more primitive than it is today.

The term business intelligence was coined in the 1950s and then reinvented in the 1980s; however, it has described a major category only from the 1990s onward, specifically starting when GUIs (Graphical User Interfaces) became prevalent.. “Business intelligence” is sometimes used to comprise all of analytics; more commonly, however, it refers to tools focused on data selection and presentation.

These days, most of what we’d call BI comes in a single integrated package, focused on a dashboard; most of the exceptions are somewhat old-fashioned report writers. In the 1990s and early 2000s, however, business intelligence had several distinct subcategories …

… one of which was called OLAP (OnLine Analytic Processing). Actually,  the term “OLAP” has been confusingly been used to mean several different things, including:

• Pretty much all of analytics.
• A particular non-relational DBMS architecture that I prefer to refer to as MOLAP (Multidimensional OLAP).
• An integrated suite of DBMS, 4GL, and perhaps other tools around a MOLAP architecture. (Examples: The IRI Express and Arbor/Hyperion Essbase products Oracle bought.)
• A client-side BI tool with a little MOLAP DBMS built in. (Example: Cognos’ erstwhile flagship BI product PowerPlay.)

I hate the term “OLAP” with a passion, in part due to that confusion, and in part due to the specific way the confusion came about: Ted Codd introduced the term, allegedly objectively, but actually as a marketing shill for Arbor Software, which had an obvious business incentive to pretend that its specific technologies solved a broader class of problems than they actually did.

OK. With that too cleared away, I feel ready to write about the actual history of analytic technology.

• Oracle started out doing what amounted to custom development for government (military/intelligence) clients.
• Even when Oracle said it had productized its software, the stuff didn’t work very well without services to get it running.
• Oracle and Ingres both got a huge fraction of their early revenue* from deals to port their software to various brands of hardware.** That’s a lot like professional services.
• Oracle’s huge Tools Group grew out of professional services, if I have the story straight. Indeed, its first product was written by later long-time group chief Sohaib Abbasi when he was a consultant.

*Revenue recognition rules were rather different back then. Multi-million payments or guarantees for ports could be recognized as lump-sum revenue up front.

**Ingres once ran on more hardware platforms than it had employees, when both numbers were somewhere in the 40s. Most of the boxes on which the porting was tested were in one small office.

The benefits for a young software company of being in the professional services business include:

• It can be high-margin, especially if it shares the cost of sales with your software offering.
• It allows you to say “yes” to whatever the customer asks for. (More precisely, it takes you closer to that goal that you’d be without a service offering.)
• It allows you to fund capable staff. Or to put it even more bluntly …
• … professional services brings in revenue that pays your bills.
• It gets you involved with customers, learning stuff about their needs, and specifically addressing them.

The disadvantages of professional services generally boil down to various forms of defocus; you can screw up your development schedule, your development priorities, your sales priorities, your partnering efforts, your market positioning, your burn rate or just about anything else.

Many software companies pursue substantial professional services when they’re young. Many don’t. Both strategic choices can make sense.

This was all slightly before my time — I only entered the industry and met Software AG in 1981. So does anybody else out there recall more of the story than I do?

Here’s the part I know from my own memories as

• The son of a Management Horizons employee (namely my Dad).
• A software industry stock analyst (in particular, one who followed Informatics General).

My father moved to the Columbus area in 1973 to join Management Horizons, a consulting firm serving retailers. Management Horizons had its own spin-out already, a time-sharing company called Management Horizons Data Services (MHDS), with which it still shared a building on what is now Old Henderson Road in Upper Arlington. And, this being a world full of coincidences, MHDS is very on-topic for the primary focus of this blog (software industry history).

MHDS’ main business was a full suite of what we might now call ERP for distributors and/or retailers. That never amounted to much. But its secondary business was an electronic interchange for direct placement of orders, called Ordernet. Ordernet turned into Sterling Commerce, a > $1/2 billion company that has been acquired for >$1 billion more than once.

The chain of events, roughly, is: 

• Management Horizons sold MHDS to what would become Citibank.
• Citibank flipped MHDS to early software industry conglomerate Informatics General.
• Sterling Software grew itself 10X in size by a hostile takeover of Informatics. (Sam Wyly was involved in that.)
• Sterling split into Sterling Commerce and what I might call The Rest of Sterling.

At that point I forget the details, but a couple of multi-billion dollar acquisitions/divestitures have ensued.

Some day I may dig out the numbers for Informatics’ revenue breakdown in the early 1980s. Ordernet wasn’t big, but looked like a rising star. MHDS  classic wasn’t big either, and didn’t look like it was going anywhere. Both appearances were later born out.

Online research uncovered some other sources, namely:

• A two-part biography of Alton Doody’s Management Horizons co-founder William Davidson, whose main relevance to this post is that it pointed me to …
• … a 1996 history of Management Horizons written by Davidson.
• An oral history from Informatics co-founder Werner Frank.

On pages 18-19, the Werner Frank link notes that Informatics acquired MHDS in what sounds like 1974, from then-owner Citibank. MHDS was doing $7.8 million in revenue (which Frank failed to break out among its two business segments), was acquired for $3.4 million, and apparently brought a $3.3 million long-term contract along with it, as well as a cheap loan.

Unfortunately, Davidson’s history* is inaccurate in some details I can check from memory. E.g. it says the Doody Company was created in 1974, which is too early, and it omits Informatics General from the history of MH’s information processing division. It also includes Aetna in the list of outfits that owned MHDS, which is news to me and also, apparently, to Werner Frank.

*As I write this, another copy is available for sale on Amazon. Just Google on “History of Management Horizons.” Yes, we paid around $20 for my copy too, even though there’s a $0.99 price tag on it clearly crossed out.

Anyhow, Davidson wrote:

• One of Management Horizons’ founding projects was helping retrain the NCR sales force away from dumb cash registers and toward what we’d now call point-of-sale devices, a key step in the IT revolution of actually using POS information.
• MHDS started out with two leased IBM System/370-155 computers.
• MHDS was divested to Citicorp’s predecessor bank in May of 1973. “At the time of divestiture some 92 wholesale distribution sites were under contract to MHDS with about 30 or so on-line to it.”

• Mature financial management arrived in the person of Jeff Henley.
• A serious commitment to product quality emerged, often credited to the late Bob Koii.
• Mike Fields* brought enterprise-class adult supervision to Oracle’s US field operations.

Political battles still raged at Oracle — Mike Fields vs. Craig Conway, Terry Garnett vs. Jerry Baker, and later on Mark Benioff vs. pretty much everybody. But the company was ready to move to next level. I personally bought Oracle stock around that time, and it grew into a rather significant fraction of my total IRA.

*For a guy who

• Made his name kicking Cullinet’s butt as New England regional sales manager for Applied Data Research,
• May never have hit another sales target again, and
  
• Greatly improved his most famous employer by being fired from it,
  

Mike Fields has a lot of fans — including me. Some people actually do deserve to fail upwards, and I think he was one of them.

The person who made Oracle into a truly major enterprise technology company, however, was nobody I mentioned above. Rather, it was Ray Lane. Some of the reason, as I’ve already noted on DBMS 2, was general good management. Ray was also a tireless salesman, for example once telling me of a very large deal he won by being the only executive of his seniority to show up and give a pitch. But the most interesting part of his success was the strategy he fostered, namely the integration of packaged software and high-margin professional services.

In theory, that integration is blindingly obvious.

• If your professional services are rich enough, you never have to say “No” to a product feature query. You only have to state a price.
• You’re selling two high-priced things in one sales cycle.
• Since your professional services people are* the world’s greatest experts on your products, the services can be inherently high-price/high-margin.

*or at least can be represented as

In practice, despite many attempts, it had rarely worked well before. Reasons included:

• Packaged application software products weren’t actually very easy to customize and extend.
• The core competencies of application development /execution software companies and applications purveyors were very different.
• Even application software companies and application-oriented professional services companies had different cultures.

Some of the few successes were:

• American Management Systems, which did a great job selling semi-custom software, especially to governments and other not-for-profit customers.
• To some extent, vertical market providers in highly regulated industries (not-for-profit, banking, insurance, health care).

Apparently, highly regulated customers felt they had to have what they had to have, and paid whatever price was necessary to get it.

I haven’t identified a silver-bullet insight that Ray Lane’s Oracle had that other companies didn’t, allowing them to succeed where others had failed. They just executed and got it in large part right. Indeed, their technology for semi-custom applications went just as awry as most other vendors’, most notably in an effort to provide industry-specific I-CASE (Integrated Computer-Aided Software Engineering) models as project starter kits. Even so, the business model just seemed to work.

In large part, the industry followed suit. To this day, professional services are a bigger part of software vendors’ business mixes than they were before Ray Lane.

One might ask, “What about all those system integration partners who recommend software, and don’t want to be competed with?” Those came to the fore in the 1990s as well, after Rob Kelley and Brian Sommer paved the way at Andersen Consulting (now Accenture) in the previous decade. SAP owed a large part of its success to them, as did Siebel Systems, whose early success had a lot to do with a super-close Andersen relationship. Indeed, by all signs they are a major reason Oracle retreated from Ray’s strategy. Services (mainly consulting), which hit 58% of revenue in FY 1999, were down to 21% a decade later.

Other possible reasons for abandoning its successful strategy include:

• Oracle soon embarked on a strategy growing by acquisition.
• Geographic outsourcing would have made things harder to manage.
• It’s hard to scale a professional services business too big while maintaining premium quality.

But those are subjects for another time.

Ingres and Oracle were developed around the same time, in rapidly-growing startup companies. Ingres generally was the better-featured product, moving a little earlier than Oracle into application development tools, distributed databases, etc., whereas Oracle seems to be ahead on the most important attributes, such as SQL compatibility — Oracle always used IBM’s suggested standard of SQL, while Ingres at first used the arguably superior Quel from the INGRES research project. Oracle eventually pulled ahead on superior/more aggressive sales and marketing.

Then in the 1990s, Ingres just missed the DBMS architecture boat. Oracle, Informix, Microsoft, and IBM all came out with completely new products, based respectively on Oracle + Rdb, Informix + a joint Ingres/Sequent research project, Sybase, and mainframe DB2. Ingres’s analogous effort basically floundered, in no small part because they made the pound-wise, penny-foolish decision to walk away from a joint venture research product they’d undertaken with innovative minicomputer vendor Sequent in the Portland, OR area.

Computer Associates bought Ingres in mid-1994, and immediately brought me in to do a detailed strategic evaluation. (Charles Wang telephoned the day the acquisition closed, in one of the more surprising phone calls I’ve ever gotten, but I digress … Anyhow, the relevant NDA agreements, legal and moral alike, have long since expired.) There was nothing terribly wrong with the product, but unfortunately there was nothing terribly right either. Aggressive investment — e.g., to get fully competitive in parallelism and object/relational functionality, the two biggest competitive differentiators in those days — would have been no guarantee of renewed market success.

Notwithstanding the economic question marks, CA surprised me with its enthusiasm for taking on these technical challenges. But another problem reared its head — almost all the core developers left the company. (If you weren’t willing to sign a noncompete agreement that was utterly ridiculous in those days, at least in the hot Northern California market, you couldn’t keep your job post-merger.) And so, like almost all CA acquisitions outside of the system management/security/data center areas, Ingres fell further and further behind the competition.

Some of the same information made it into my post here on Ingres history later the same year, but for some reason not all did.

Folks, QUEL was the original language for Postgres. And Ingres. And, more or less, Teradata.*  I’d guess Britton-Lee too, but I don’t recall for sure.

*Once upon a distant time, when I was a cocky young stock analyst, I explained to Phil Neches, chief scientist of Teradata, just why it was a really good business idea to drop T-QUEL for SQL. I doubt he was convinced quite on that day, more’s the pity.

*A blog-tagging meme is, in essence, an internet chain letter without the noxious elements.

Without further ado, the Rules of the No-Fooling Meme are:

Rule 1: Post on your blog 1 or more surprisingly true things about you,* plus their explanations. I’m starting off with 10, but it’s OK to be a lot less wordy than I’m being. I suggest the following format:

• A noteworthy capsule sentence. (Example: “I was not of mortal woman born.”)
• A perfectly reasonable explanation. (Example: “I was untimely ripped from my mother’s womb. In modern parlance, she had a C-section.”)

*If you want to relax the “about you” part, that’s fine too.

Rule 2: Link back to this post. That explains what you’re doing.

Rule 3: Drop a link to your post into the comment thread. That will let people who check here know that you’ve contributed too.

Rule 4: Ping 1 or more other people encouraging them to join in the meme with posts of their own.

Hopefully, the end result of all this will be that we all know each other just a little bit better! And hopefully we’ll preserve some cool stories as well.

To kick it off, here are my entries. (Please pardon any implied boastfulness; a certain combustibility aside, I’ve lived a pretty fortunate life.)

I was physically evicted by hotel security from a DBMS vendor’s product announcement venue. It was the Plaza Hotel in NYC, at Cullinet’s IDMS/R announcement. Phil Cooper, then Cullinet’s marketing VP, blocked my entrance to the ballroom for the main event, and then called hotel security to have me removed from the premises.

A few years later, the same Phil Cooper stood me up for a breakfast meeting in his own house in Wellesley. When one’s around Phil Cooper, weird things just naturally happen.

I got James Marsters (“Spike” on Buffy the Vampire Slayer) to autograph a shirtless picture of himself. Linda was a very serious Buffy fan, and I was no slouch in that regard myself. So we flew out to Santa Barbara to join some acquaintances from a Buffy-centric mailing list. Except first, we went – along with the hostess for the gathering – to see James perform with his rock band Ghost of the Robot in Santa Monica. She had downloaded a photo she wanted to pass out to the other (mostly female) attendees. But neither she nor Linda wanted to actually ask him to autograph it – too fangirlish or something. So I stepped up and made the request in their place. (Technically, asking James to sign anything other than a Ghost of the Robot CD was against the rules – but he was more than gracious when I said that if he signed for me it would be a great help to my relationship. )

I had (inter)national reputations in four different fields before my 24th birthday – two academic, two non-academic. I got my PhD in game theory, independently proving a theorem that was simultaneously (and a few months earlier) proved by Mertens and Neyman. Naturally, the game theory community was quite aware of my work. Then I did a post-doc in public policy for a couple of years. Some of my work – sort of a public-sector version of real options analysis, although unfortunately I wasn’t familiar with that concept – was widely circulated among the US utility regulation community. Then, at age 21, I went to Wall Street, as a stock analyst covering the software industry for a major firm (PaineWebber). That pretty much assured me of being known in both the investment and software businesses, especially as I soon got pretty high in the third-party stock analyst rankings.

Graduating college at age 16 cost me my NCAA eligibility – and I wound up regretting that aspect. On the whole, I’m hardly an athlete. But I did take a few fencing classes, and enjoyed them. So when my grad school offered an intermediate intramural fencing class, I was psyched. But when I went the first day, they explained that they didn’t have the resources to offer the class. Instead, anybody who wanted to could join and hence work out with the Harvard fencing team, whether or not they were good enough to compete. Only they couldn’t do that for me – because my NCAA eligibility had been shot when I graduated from Ohio State.

I double-dated with Larry Ellison, twice. Part of the explanation is that when I lived in Manhattan, I had almost no friends there, but quite a few in the SF Bay area. (Seriously, which kind of work acquaintances would you expect me to have more in common with – tech entrepreneurs or Wall Street professionals?) So if I wanted to introduce a new girlfriend to my friends, we’d fly out west. And if there’s one company at which I had a lot of friends, it was Oracle, which I’d first visited in 1983, when it was still located at 3000 Sand Hill Road and had fewer than 50 employees. I was very engaged with Oracle professionally through most of the 1990s – but we also got together just for fun.

I was the top-rated chess player my age in the United States. This is when I was 13 years old, in 1973. But I wasn’t nearly as good as that factoid suggests. 1973 may have been the weakest year for 13-year-old US chess players in modern times; I made #1 with only a 1650 rating. (To put that number in context, it indicates that this 2300+ rated 13-year-old could have routinely obliterated players who could, in turn, have routinely obliterated me.) More on the story may be found at this link.

I told people at a conference “I just spent the afternoon with Bill Gates’ girlfriend — and boy is my butt sore!” This was in 1985 or so. I was a stock analyst living in NYC. The annual American Electronics Association investment conference in Monterey was a must-attend event. Flight connections, however, were imperfect. So Ann Winblad had the idea that, during my several-hour layover, she’d pick me up at SFO, and we’d go drive around the mini race track that’s still there off of 101. I got pretty sore bouncing around in the little plastic cars, perhaps because I wasn’t a very experienced driver. Indeed …

… I didn’t own (or lease) a car until I was 36 years old. (To readers from California, this one may sound the oddest of all. ) I left home at age 16 to go to school in Cambridge, MA, where one doesn’t typically have a car. I left there at age 21 to move to Manhattan. Finally, when I was 36, I left Manhattan for Lexington, MA, at which point I of course got wheels. That Toyota served me well for about a decade, but eventually …

… my first car forced a DBMS vendor to evacuate its whole office building. My Toyota Camry had an engine fire in Intersystems’ building on Memorial Drive, and the building has indoor parking. The story is here. Actually, many other companies had to evacuate the same building.

My kitchen caught on fire, just as I was Twittering with LeVar Burton of Star Trek:TNG and Roots fame. That juxtaposition was a total coincidence. LeVar had just tweeted me his vehement agreement to something I said, when Linda appeared at my office door, having noticed the sound of what turned out to be a fire on the stove. This story has been the subject of several other blog posts …

By the way, what I’d said was, after LeVar tweeted his pleasure at for once actually acting again (on stage, no less), that we choose our original professions for a reason – taking them up again, even for a little while, is like going home again. Even today, I rarely feel more right than when I’m doing mathematics.

Even today, a significant amount of marketing and sales is done at vendor-run seminars in medium-quality hotels. But in those days, before the internet and hence in particular before webinars, a huge fraction of all sales cycles passed through a physical seminar-attendance step. So if you could disrupt your competitors’ seminars, you could disrupt their whole sales cycles. So M&D and MSA salespeople did just that, routinely calling hotels to outright cancel competitors’ reservations and events. If I had to name offenders’ names, I’d start with Mary Kohler at McCormack & Dodge and Roe Henson at MSA, but I’m pretty sure the men were even “worse.”*

*Truth be told, I think the whole thing was pretty funny, or else I wouldn’t be sharing it. Further, I emphatically think Mary and Roe should be admired for succeeding in what was then an extremely male world.

That story has been confirmed multiple times, with minor variations (biggest disagreement = which side started doing it first). More dramatic stories are less confirmed. My favorite of those is MSA arranging for a McCormack & Dodge contract signing to be disrupted by the M&D salesman’s arrest for delinquent child support. (Ouch!) Other confirmed examples I can think of are tame by comparison, like the blow-up dolphins the MySQL folks decorated the Sun campus with after their acquisition closed.* E.g., sending trucks with hiring or marketing messages outside your competitors’ conferences or office buildings is not very imaginative, and actually happens in lots of industries.

*Sun apparently had a major tradition of MIT-style April Fool’s pranks, one of which featured Scott McNealy’s car being stranded in – or rather on – the middle of a pond. But that’s a little outside my purview.

But one I’ve always loved is the tradition of witty product code names. Some of my favorites were from the days of the Borland/Lotus spreadsheet competition, including:

• Conan, because Borland CEO Philippe Kahn prided himself on Borland being “barbarians”
• Crom, the god Conan prayed to (quite so – I’ve read the books)
• Buddha, because Borland wanted to assume “the Lotus position” (one of my favorite puns ever)

Another pair comes from when John Landry, then McCormack & Dodge’s R&D chief, was developing a proprietary programming language, which he planned both to use for in-house development and to expose to users for their own customizations. (I.e., it was a forerunner of SAP’s ABAP and PeopleSoft’s PeopleTools.) The first codename was GLOP (General Language for Ordinary People). That was eventually replaced by SLOB (Simple Language for Ordinary Bozos). To the best of my knowledge, those code names never made it into any actual product documentation.

I think I’ll stop there. I do have other stories of wise-assery I could add, but I think I’ll hold them back until I’m ready to take the time to wrap them in a bit of context …

Edit: Maybe I’ll add more here as I think of them.

• Larry Ellison, Mitchell Kertzman, and David Roux did a hilarious site spoofing the dotcom bubble. Unfortunately, it only persists in incomplete Internet Archive form, but that’s enough to show the key point.

That thrilling advance notwithstanding, I’m not aware of much practical significance to any TPC-H-related DBMS product development. But multiple people this week have reminded me this week the TPC-A and TPC-B played a much greater role spurring product development in the 1990s.  And I indeed advised clients in those days that they’d better get their TPC results up to snuff, because they’d be at severe competitive disadvantage until they did.

It’s tough to be precise about examples, because few vendors will admit they developed important features just to boost their benchmark scores. But it wasn’t just TPCs — I recall marketing wars around specific features (row-level locking, nested subquery) or trade-press benchmarks (PC World?) as much as around actual TPC benchmarks.  Indeed, Oracle had an internal policy called WAR, which stood for Win All Reviews; trade press benchmarks were just a subcase of that.

And then there’s Dave DeWitt’s take.  Dave told me yesterday at SIGMOD that it’s unfortunate Jim Gray-inspired debit/credit TPCs won out over the Wisconsin benchmarks, because that led the industry down the path of focusing on OLTP at the expense of decision support/data warehousing.  Whether or not the causality is as strict as Dave was suggesting, it’s hard to dispute that mainstream DBMS met or exceeded almost all users’ OTLP performance needs by early in his millenium. And it’s equally hard to dispute that those systems* performance on analytic workloads, as of last year, still needed a great deal of improvement.

*IBM’s DB2 perhaps excepted. And I say “last year” so as to duck the questions of whether Exadata finally solved Oracle’s problems and whether Madison will once Microsoft releases it.