« 1.019: (halfway) around the world in (a hundred and) 80 days | Main | 1.021: bass-ackwards marketing »

August 15, 2008

1.020: how much does a "free" xiv array really cost?

The silence from IBM on their stealth XIV announcement this past week has been deafening. And I can assure you that IBM PR would be embarrassed to know how many people have read my XIV post expose since I published it Tuesday afternoon. The traffic to my blog has been at an all-time high this week.

One reader picked up on the bullet about the power overhead of the XIV's one-size-fits-all 180 drive standard configuration, and he sent me theFlorida's new electric chair calculated annual costs for power and cooling of a single XIV frame (based on the specs posted by IBM).

This got me thinking. As many of you know, Moshe and his new-found IBM sales team have been visiting virtually the entire EMC customer base offering them "free" XIV systems for evaluation.

And I know that at least some of my readers have taken advantage of the free storage offer - it's hard to turn down "free" in today's economy.

But if you're one of them that did, you might want to sit down before I go on....

Oh, sorry - don't sit over there. That chair is for the EMC sales reps that jumped ship for XIV. Let's find you a more appropriate seat... over here in the corner looks better. The Comfy Chair

There now - are you comfortable?

That's good.

Because what you probably didn't realize is that the "free" 180-drive XIV array that IBM is offering you to evaluate will actually cost your company more than $20,000.00  a year in electricity alone to operate and cool.

That's pretty outrageous for a measly 80TB of usable capacity, don't you think?

Thankfully, there aren't all that many of you that got sucked into this marketing ploy. And hey, it wasn't as Deceptively Delicious as Microsoft's Mojave stunt now, was it? I mean, the IBM rep informed you up front about the power costs, right?

And why the heck did TonyP bring Mojave up in his blog this week in the first place?
Wasn't there anything significant happening at IBM this week that he could blog about?
Did he too miss the XIV announcement, like the rest of IBM Marketing?

What's that?

You want me to stop rambling for a minute while you run off to unplug that XIV space heater before your boss finds out you've been wasting so much money and energy?

No worries, I'll wait. Meet you back here after the break...

 

xiv proves that it isn't easy being green

OK, before we go on, I have to admit that I haven't personally measured the power utilization of an XIV v2 box from IBM, and that there are probably few, if any, IBM customers prospects who have one of this new vintage XIV's running in their shop.

That said, IBM's posted materials describe the power requirements for the standard configuration XIV Storage System as follows:

Operating environment
Temperature:  10 to 35 degrees C
Relative humidity:  25 to 80  percent
Max wet bulb:  23 C
Thermal dissipation:  26K BTU/hour
Maximum power consumption in watts:  8.4 KW
Sound Power, LwAu = 8.4 bels

At a cost of $0.15US per KWh, a system consuming 8.4KW/hr and dissipating 26K BTU/hr will cost somewhere between $20,021 and $21,807 to operate and cool in a reasonably efficient data center. And at only 80 TB usable, that works out to more than $250.00 per usable terabyte -- per year!

I'm reminded of The Inconvenient Truth of Al Gore's carbon footprint.

To put these costs in perspective, I created the following table comparing the annual power costs of the XIV SSv2 to various Symmetrix and CLARiiON configurations. All EMC configs include minimum required spare drives and overheads (vault space, etc), and all power requirements include both consumption + cooling costs, cooling costs calculated from rated BTUs dissipated (where available) or as approximately 1x the power consumed.

System Disk Configuration Usable Capacity Annual Power Cost @ $0.15/KWh Annual Power Cost / Usable Terabyte
IBM XIV SSv2 180 1TB SATA XIV RAID X 80TBu $20,021 $250.56/TBu
EMC
DMX4-950
180 1TB SATA
RAID 1
~85TBu $14,614 $171.93/TBu
EMC
DMX4-950
120 1TB SATA
RAID 5 (3+1)
~84TBu $11,142 $132.64/TBu
EMC
DMX4-950
104 1TB SATA
RAID 6 (14+2)
~84TBu $10,461 $124.54/TBu
EMC
CX4-120
5 400GB FC
115 1TB SATA
RAID 5 (4+1)
~82TBu $8,629 $105.23/TBu
EMC
DMX4-950
180 1TB SATA
RAID 5 (3+1)
~123TBu $14,614 $118.81/TBu
EMC
CX4-240
6x450GB FC
174 1TB SATA
RAID 5 (4+1)
~123TBu $12,499 $101.62/TBu

There are several interesting observations that can be made from this table. Obviously, IBM's XIV is the most power-hungry system in the bunch - by far - even compared the DMX4 using mirrored drives (2nd row down).

And since the EMC arrays all support RAID 5 and RAID 6 configurations, EMC customers can not only get the same usable capacity as the XIV using fewer drives (rows 3-5), the energy costs per usable terabyte is less than half that of the XIV!

Finally, as the last two rows show, both the DMX4 and CX4 can deliver more usable capacity using the same 180 disk drives as the XIV, while still using far less power!

And, as I pointed out before, if you choose to take advantage of IBM's innovative "capacity on demand" pricing strategy for the XIV, you'll perhaps save some money on the capacity that you're not using, but you'll still be paying the same electric bill for all 180 drives in the system!

so let's review

Combining the observations from my previous posts on XIV, Moshe has built sold IBM a storage array that:

  • looks a lot more like it was intended to compete with a 1999 Symmetrix 8000 series than against a 2008 Symmetrix DMX-4 (or even CLARiiON CX4, for that matter),
  • delivers less than half the usable capacity of the drives a customer purchases,
  • runs with zero protection for cache against failures or errors, <corrected Aug 31, 2008>
    mirrors write data over a measly 1Gb Ethernet interconnect to a second node
     
  • uses that same 1Gb Ethernet interconnect to deliver data to 4Gb Fibre Channel hosts,
  • is limited to the inherently slow read miss performance of 7200 rpm SATA disk drives
  • has no long-distance asynchronous replication capability,
  • offers no upgrade path to past or present customers,
  • cannot even non-disruptively upgrade its own firmware,
  • comes in 1 and only 1 capacity configuration,
  • has zero track record running mission-critical tier 1 or 2 applications in production,
  • and consumes more power per usable terabyte than any of the target competitive offerings from EMC!

Those are some pretty challenging facts for a new storage wannabe product to overcome, if you ask me.

So the next time Moshe or your IBM rep drops by to offer you a "free" eval XIV unit, you should probably think twice before you accept. Otherwise, your boss might be putting YOU in the "special" chair when they get the electric bill!

And you know - IBM might want to stop calling this new system "XIV" altogether- and not just because the name isn't unique.

No, perhaps they should resort to calling it simply "Fourteen."

At least that way it would rhyme with green!


TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d834c659f269e200e553e84de88833

Listed below are links to weblogs that reference 1.020: how much does a "free" xiv array really cost?:

» Recent Links Tagged With Xiv from JabberTags - Find New Sites and Explore the Internet
Bookmarked your site as xiv at JabberTags! [Read More]

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Stewey

barry, how likely is it that emc customers would purchase an array of just 180 drives? Isn't a DMX-4 950 broken out into quadrants of 120 drives each? Which would make a similar dmx config more like 240 drives, in a mirrored config. Also, this would be presented in multiple cabinets. While the usable capacity would be greater, it would also take up a larger footprint. Was your calculations based on the additional cabinett (even with the 180 drives)?

DenisG

What will be the next IBM acquisition?

Looks like XiV was a bust.
http://blogs.eweek.com/storage_station/content/why_ibm_is_regretting_300m_xiv_acquisition.html

the storage anarchist

Stewey -

The DMX4-950 supports up to 120 drives in the main bay, and optionally up to another 240 in an add-on drive bay (the 240 are "daisy chained" to the first 120, not controlled by separate add-on controllers). And yes, the calculations included that second cabinet.

180 drives is exaclty half the maximum number of drives the 950 can support, and in fact is a popular configuration for people who want expansion space for the future.

And perhaps you don't understand the configuration flexibility of Symmetrix, but drives can be added to any Symm in practically any increment - you don't have to totally fill the drive pods for a practical and viable machine. You could actually have a 170 drive system if you'd like, or even a 190 drive system - something you can't do with an XIV. You can mix-and-match RAID protection to meet your cost, performance and avaialbility objectives - another thing the one-size-fits-all XIV does not offer. In fact, the majority of DMX systems operating today employ multiple RAID types and multiple drive speeds & capacities - not possible with XIV.

And had I used the 2-bay DMX4-1500 instead for the 180 drive config, it would have come in even lower $'s/TB for power. That system starts out with 2 cabinets and a max of 240 drives, but it would have fewer unused drive slots and more fully utilized power supplies, for even better $/GB.

But since the 120-drive 950 delivers more usable capacity in a single cabinet with fewer drives and optimally loaded power supplies, I decided to stick with the 950 for all the comparisons - even though they aren't all the "best" possible configuration.

AndyRH

There are goods and bads. The 180 disk thing seems to really mess some people up, but the idea is to scatter write the data and have more disks working on the problem. This allows a better cost point because big SATA drives are cheaper than fast FC drives. We may test one next year to see if the performance claims are even close.
And remember global cooling is part of global warming, now drink your kool-aid.

the storage anarchist

Andy -

You'll notice that I've been comparing the XIV to SATA drive configurations for both CLARiiON and Symm, so the cost comparison of SATA vs. FC is irrelevant. All practical storage arrays support SATA today (the DS8000 being the last to join the party), so that's not really an XIV differentiator.

And with EMC Virtual Provisioning on either CLARiiON or Symm with 180 SATA drives, you can expect to get the same "wide striping" performance benefit of the XIV. BUT - you'll get more than 50% more usable capacity, making either EMC offering even more cost-effective than XIV.

I think we're all looking forward to actual performance data, though. Given the fact that a 7200rpm SATA drive connet respond to an IO request any near as fast as a 15K rpm FC drive, it should be interesting to see the results of real-world workloads on XIV systems.

SRJ

Barry -

"I think we're all looking forward to actual performance data, though."

Dude...SERIOUSLY?! Did you ACTUALLY just say that?

C'mon man... I realize it's hard for FUD Factories to be consistent, but give us a break! What kind of "actual performance data" would you like to see?

I'm sure IBM (not to mention all of us!) would LOVE to see some "actual performance data" for ANY of EMC's gear.

Ever hear of the SPC? Oh yeah - you won't stop whining about how irrelevant "actual performance data" is!

Seems like every major storage vendor on the planet is willing to provide us with some sort of standard, independently verified "actual performance data" ....except EMC!!!

With all due respect - put your money where your mouth is, Barry! Maybe then someone will take you for something other than a tool for the marketing department.


EY

This article is very biased towards EMC.
First of all why does the article covers only EMC and not Netapp,EVA,DS...

In the field - EMC doesn't like to sell SATA and will not recommend that so you will need to compare XIV to FC solution which means more disks "300FC or 400FC" - so in order to get the same capacity with the same performance you will need twice the disks in EMC configuration - this means it is definitely not "Greener" than XIV.

More issues that are not considered here is that XIV use much more cache on regular basis which means it is using the "disk" less on average - this is one of the thing that allows
XIV to show the same performance (and sometimes better).

Also XIV allows taking much more than 8 snapshots on their disks while EMC allows max of 8 snapshots on single lun and require to perform a clone in order to extend the number of
Snapshots beyond 8. "clone" means much more disks

Part of the points that are mentioned by the article author are correct (no-long asynchronous replication, firmware/software upgrade performed offline) but these issues are being addressed by XIV and should be resolved in the next few months


No doubt that XIV isn't perfect and lack some features because of it's young age but from the "green" side it is probably greener than EMC

the storage anarchist

SRJ -

"Actual performance data" as in "real-world workloads."

I indeed maintain that the SPCs are meaningless for their purported purpose of helping to choose a storage platform - it is anything BUT real-world, as its originators will tell you (off the record, of course). Testing how a system operates at the knee of the stress curve tells you nothing about how well that system operates under normal workloads.

No, what I ask for is real-world reports. Oddly, I haven't found a single customer willing to provide actual data on XIV performance - nothing more than "it's fast." For some reason, nobody has been able to quantify it beyond that...

the storage anarchist

EY -

EMC sells SATA-based systems all the time, both Symm and CLARiiON, so the apples-to-apples comparison is SATA vs. SATA.

Using disk less on average can't compensate for the fact that SATA drives are slow, so when you DO go to disk, response times are huge relative to FC drives. Today's working sets routinely exceed the cache size of even the largest cached arrays, such that fast disks (and flash drives) are necessary to meet performance objectives in many cases.

And when the cache has to be accessed over the XIV's slow 1Gb/s Ethernet internal interconnect, I suspect that the latencies really start to pile up.

Your EMC snapshot info is woefully out of date -today's DMX systems support 128 snaps, and CLARiiON supports similar numbers of space-saving snaps.

But you are correct about one thing - it would be nice if someone could explain how the NS, DS and XIV families overlap. IBM's marketing materials for XIV position it as a replacement for practically every storage array on the planet - EXCEPT the ones IBM sells (including NetApp).

Rather disingenuous, I say.

SRJ

Barry,

Slight correction on your statement about protection for cache:

According to the "Theory of Operation" guide, incoming data is written to two separate caches (in two separate modules) before being written to disk.

SRJ

XIV has a case study for a banking customer (Bank Leumi) who runs 7 (seven) Nextra's in production (first one since 2Q2006!). Everything from Tier 1 Oracle workloads, all the way down to archival storage.

Your comment about not having any track record seems like a silly error in light of the freely available case study on XIV's web-site.

Storage News

1) 1) You forgot how EMC was selling in the first years with the "try and buy" concept.
2) “And when the cache has to be accessed over the XIV's slow 1Gb/s Ethernet internal interconnect, I suspect that the latencies really start to pile up.” The internal data bus is double PCI-X which is faster than FC and not 1Gb/s Ethernet. Your “latency story” is not true.
3) Are CLARiiON CX-4 and DMX not overlap?

mox3311

You people crack me up. Barry, please, your behavior is that of one who never had a catch with daddy. Would you please stop, please. EMC is not innovative in anyway, period. EMC has built its glutinous empire through acquisition, big deal. Your innovation left in 2002 and is now poised to come after you, and you know it. The XIV is going to morph and develop. EMC has one motivated Marine, or tank commander, on its heels, and your comments are leaking your concern. Good luck.....

the storage anarchist

I stand corrected, and I've updated the post regarding cache write mirroring...writes are indeed copied to a second module before they are acknowledged.

And indeed, they are mirrored over the 1Gb ethernet, and not over the PCI Express bus, so Ethernet latencies are indeed a factor.

the storage anarchist

mox3311 - You're pretty funny yourself.

Next time you see Moshe, ask him exactly when it was that he was a tank commander. I'm pretty sure that it never happened - I've heard it was an entirely fabricated story, invented by an EMC Marketing executive who thought he needed a more colorful history.

Rob

Barry,

I had suspicions about your numbers and never got around to digging them up.

Focusing on one of your configs above (I could dig through the others, but if one is broken, the others are suspect) - I'm seeing much higher power usage than you are suggesting.

The 180 TByte mirrored DMX-4 950 uses over 36000 BTU/hour, 9.4 kw (power factor .9).

I'm pulling that from here:
http://www.bull.com/catalogue/download/sdmx4950_va.pdf

the storage anarchist

Rob -

Several readers have pointed this out to me, in several different forums (sic?).

Unfortunately, you've all misunderstood the specifications.

A spec sheet, such as the one you pointed to above, is required to present the MAXIMUM amount of power that a device will require, in its MAXIMUM configuration.

In the case of the DMX4-950, the spec you quote is for a fully-configured 2 cabinet system with the maximum configuration of 360 of the most power-hungry disk drives in the fleet (the 73GB 15K rpm drive, I think). The spec also includes not only the normal operating-under-load power draw, but also the power required to recharge the integral standby batteries at the same time.

As you probably know, disk drives themselves represent most of the power utilization and heat generation in a storage array (also most of the space and weight). But cutting the number of drives in half doesn't halve the power - the UPSs, power supplies and logic boards are pretty much constant in this case - most of the drive bay infrastructure is the same for 60 drives as it would be for 240 drives in that bay.

I assure you that my power numbers were accurate for all of the EMC configs, and were not faked in any way. The comparisons were created using the EMC Power Calculator, which is available to customers on PowerLink. This tool allows for precise power prediction for any given EMC storage or SAN product, and is based off of ACTUAL MEASUREMENTS taken with different configurations. And yes we have an internal-only version that we use to calculate competitor products - also based upon actual measurements (whereever possible).

This tool is highly accurate as the folks over at WikiBon can attest - they are using it to help EMC customers take advantage of PG&E's power rebates.

And by the way - I did indeed use "maximum power" for the comparisons, which includes some overhead (like the battery recharging) that isn't necessarily be required all the time. As the folks at Wikibon pointed out to me, I probably should have used "nominal" power instead of "maximum". However, based on IBM's documentation for the XIV, the "nominal" power advantage of the EMC products would have been even greater - significantly so in the case of the DMX products.

But since spec sheets rarely include "nominal" power. So I've left it as-is...for now.

Hue

EMC Marketing cracks me up. I have been a storage admin for too long. SATA+EMC = Much pain. Symm is an awesome box but it is designed for FC drives. We just evaluated the XIV. For the cost/simplicity/reliability and performance-by-cost the XIV boxes hold up. If I can convince my director to give me the funds I would do it in a heartbeat.

Berry (aka EMC marketing should have never stated on this thread).

You guys are better of acquiring Lefthand of someone else that can plan ball on the same field.

The days of overpriced storage are coming to an end. I just can't believe its IBM that brought it to the Market. Go figure no wonder there stock is still holding up.

March,3rd,2009
-HUE

The comments to this entry are closed.

anarchy cannot be moderated

about
the storage anarchist


View Barry Burke's profile on LinkedIn Digg Facebook FriendFeed LinkedIn Ning Other... Other... Other... Pandora Technorati Twitter Typepad YouTube

disclaimer

I am unabashedly an employee of EMC, but the opinions expressed here are entirely my own. I am a blogger who works at EMC, not an EMC blogger. This is my blog, and not EMC's. Content published here is not read or approved in advance by EMC and does not necessarily reflect the views and opinions of EMC.

search & follow

search blogs by many emc employees:

search this blog only:

 posts feed
      Subscribe by Email
 
 comments feed
 

 visit the anarchist @home
 
follow me on twitter follow me on twitter

TwitterCounter for @storageanarchy

recommended reads

privacy policy

This blog uses Google Ads to serve relevant ads with posts & comments. Google may use DoubleClick cookies to collect information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide ads about goods and services of interest to you. If you would like more information about this practice and your options for not having this information used by Google, please visit the Google Privacy Center.

All comments and trackbacks are moderated. Courteous comments always welcomed.

Email addresses are requested for validation of comment submitters only, and will not be shared or sold.

Use OpenDNS