Tag nosql

Is it Time For a DBMS Mass Extinction?

On Sept 19, 2014, InfiniDB (formerly Calpont) announced it was closing its doors after failing to secure financing to continue operations. Establishing differentiation in a crowded market and competing over a finite supply of large enterprises eroded InfiniDB’s position. It’s easy to think this is an isolated story of a single company that failed to gain traction. However, I think this is just the first sign that the asteroid is coming.

The Permian Extinction—When Life Nearly Came to an End
Source: NatGeo – The Permian Extinction—When Life Nearly Came to an End

Over the last eight years or so, dozens of new vendors have emerged offering specialized types of DBMSs. The website nosql-databases.org tracks about 150 different DBMSs. As I see it, the current level of diversity in the DBMS space is simply unsustainable.

This won’t be a popular view. After all, several reasons have been given for the explosion of DBMSs. Most of the reasons are thinly veiled market positioning from vendors desperate for market share. Market positioning reasons almost always talk about how the “old” rules of data management no longer apply. And when people say the old rules no longer apply, you’re in a bubble.

Open source is another element driving DBMS diversity. Open source licensing allows independent developers or companies to derive new products from OSS staples like MySQL and PostgreSQL, or combine several projects into entirely new offerings. To some extent, this is facilitated by sites like github. These sites provide a virtual water cooler to develop ideas and features without needing to get together in meatspace. Redis is a great example of the power of virtual collaboration and development.

Another reason given for the diversity of DBMS models is the growth of data volumes and varieties. This argument looks much like the first: legacy DBMS vendors simply can’t cope with new data demands. To a certain extent this reason has some merit, but not nearly enough to justify the continued existence of dozens of DBMS vendors. And don’t forget the resources these vendors have. When the asteroid strikes, they’ll be hiding under piles of money.

If the old rules still apply and the data expansion argument has questionable merit, what has supported the number of DBMS vendors entering the market? Other than loads of VC funding, the answer appears to be a simple one: hardware.

Computing hardware is always increasing in capability and decreasing in cost. But it’s uncommon for processing, storage and networking to all experience massive capability increases and cost reductions in concert. The last time it happened was in 2009. Each time this convergence happens, application developers have free reign over implementation decisions. This developer-centric approach typically lasts for a few years. After all, any code will run, and run quite well, with better hardware.

The slack capacity provided by better hardware might make you think you can do things you wouldn’t previously consider. The old rules may no longer apply. There might even be a free lunch!

Applications are developed and some value is created, but the result is the proliferation of data silos at the cost of abandoned information governance.

Information management realities always assert themselves. Data in silos is fine for systems of innovation because you’re only focused on data use. But those systems eventually become systems of record, where data reuse is paramount. Systems of record must have capabilities of description, organization, governance, integration, among others.

When the asteroid strikes, it won’t be a fiery rock falling from the sky. It will be IT Ops reasserting the need for adult supervision. DBMSs providing the required capabilities will likely thrive, while those that don’t, won’t.

And not even Bruce Willis can save vendors from IT Ops.

Thanks to Merv Adrian for reviewing and contributing to this post.

NoSQL for Competitive Hiring Advantage

Regardless of your location or industry, hiring technical talent is challenging. This is particularly true in areas like Silicon Valley, where 59% of employers plan to add workers. Companies typically resort to  blunt instruments, like larger salaries or signing bonuses, to bring talent in the door. (The merits of this strategy are debatable.) Some companies are thinking beyond these basic incentives to attract workers. These companies are exploring how adopting new technologies can attract scarce development talent.

This is counter-intuitive. After all, you should only adopt a technology if it solves a business problem. Does it save money, increase revenue or capture some competitive advantage? If so, adopt. Otherwise, avoid. The flaw here is hiring is a business problem. Companies have to attract and retain talent to enable competitive advantage.

Doesn’t this sound familiar? “We have a problem. Let’s throw technology at it.” Now that type of thinking is expanding to hiring. It’s the technology trap all over again. If your company isn’t an appealing place to work, technology won’t fix the problem. It only delays meaningful change to your organization.

Looking to work at one of the best companies in the world? The Information Management team is hiring in North America and Europe. Leave a comment (I won’t publish it) for more information.

NoSQL Shouldn’t Mean NoDBA

Last September I conducted an informal survey of NoSQL adopters to improve our understanding of who is using NoSQL and why. The results were largely what I expected, except for the respondent profile. Database administrators (DBAs) appear to be significantly underrepresented in the NoSQL space, representing only 5.5% of respondents:


The possibility of selection bias occurred to me, but these numbers look accurate based on conversations with clients, vendors and developers. DBAs simply aren’t a part of the NoSQL conversation. This means DBAs, intentionally or not, are being eliminated from a rapidly growing area of information management. Application developers may be getting what they want from NoSQL now, but cutting out the primary data stewards will result in long-term data quality and information governance challenges for the larger enterprise (see “Does Your NoSQL DBMS Result in Information Governance Debt? (G00257910)).”

If you’ve adopted a NoSQL DBMS, what are your integration and governance plans? Do you even have any?

Which Database are You Taking to Prom?

Project downloads, job postings, Github commits and mailing list activity are common methods to gauge traction for commercial and open source software projects. I frequently use those same metrics as input when looking at emerging projects, and for good reason.

Increasing traffic on a user mailing list indicates growing use and adoption. However, if that traffic continues growing, especially after the milestone release, it can indicate poor documentation or quality control. Do the same questions keep coming up on user forums? Your documentation is poor or nonexistent. Did the bug count go up after the 1.0 release? Looks like you’re focusing on features over quality.

Metrics like these are useful but they don’t exist in a vacuum. They need context, the type provided by the overall market, vendor strategies and customer feedback. A simple measure of popularity lacks this context, and therefore lacks insight. Additionally, popularity doesn’t measure sentiment. You don’t know if someone on Stackoverflow is saying “Database X is awesome,” or “Database Y is awful.”

Also concerning is the self-fulfilling notion of popularity. Once a database is measured as relatively more popular than another, it will experience a further uptick in popularity metrics and the cycle continues. Ultimately this results in more marketplace hype and confusion.

Copyright © Nick Heudecker

Built on Notes Blog Core
Powered by WordPress