0.076: oops!... i(bm) did it again!
I know that many of you are getting tired of me pointing out the frequent faux-pas made by competitor's executives. To you, I apologize in advance for today's post, and I'll understand completely if you skip this entry or unsubscribe from my feed in protest.
Especially those of you from IBM, on both sides of the pond. I seem to get more hits from the ibm.com domain on these articles than from anywhere else!
I am sincerely trying to stop, honest I am. But just I can't. At least not until these guys stop feeding me material.
Before I continue, though...a note about today's theme.
About a year ago, when I was first thinking of starting up this blog, Chuck Hollis told me one of his super-secret tricks for attracting hits to his blog. He said he would include "Britney Spears" as one of the keywords for every one of his posts, and that he'd get a sizable percentage of hits from search engines like Google and Yahoo! as a result. Not that I need the hits, but the song title fits my topic, so I figured I might try it to see what happens.
So, if you're a Britney fan who accidentally got lured here by this little ploy, my apologies to you as well. This probably isn't what you were looking for.
But if you're both a storage geek and a Britney fan - Welcome! You'll probably recognize the subtitles below...
what u see (is what u get)
Today's poster children for the hoof-in-mouth charity are none other than IBM's general manager of storage, Andy Monshaw himself, and Clod Barrera, IBM engineer and chief technical strategist.
As Dave Raffo points out in his article over on SearchStorage (Blues Clues: IBM's storage roadmap...), Andy apparently waxed enthusiastically about "emerging technologies" in the storage space during his talk at SNW this week.
What was it that so excited Andy? Thin Provisioning, Deduplication, Flash Drives and MAID.
ALL of which have been available from multiple other vendors for a couple of years now. (OK, ALL except for Flash - but more on that later).
None of which IBM is shipping today.
And none of which Andy actually "announced." Or even implied was coming soon.
And then Andy disappeared, leaving poor Clod Barrera to suffer the slings and arrows of the audience.
And with that little interaction, I think Andy made it clear that IBM has proudly claimed sole possession of last place in the storage technology race.
But al least they're thinking about this exciting "new" stuff that others have already implemented.
where are you now?
That's probably what Clod was thinking after Andy left him to answer all the tough questions.
I won't rehash Dave's entire coverage of the "tough questions", but some of the responses did hit my LMAO! button again:
- Dedup - "a great idea!" yeah - that's why just about every VTL on the planet supports it already (or soon)
- Thin Provisioning - apparently only available on IBM's re-branded NetApp gear. Oh, and on the XIV platform, but "we've not yet released that as an IBM logo product," he said.
- Thin provisioning on the DS8000? - "a good expectation"
Last time I checked, you can't store all that much data on an "expectation." Reportedly access times are extremely elongated...
(i can't get no) satisfaction
Actually, *I* can. And did.
Because Clod pretty much confirmed what I've been saying for months - that the DS6000 is all but dead.
OK, what I actually said is that it WAS dead, but when TonyP piped in to assert that I had misinterpreted the IBM Alert, I (almost) believed him.
But Clod made it clear that my premonition was correct, if perhaps a bit temporally ahead of reality (emphasis mine):
Barrera also indicated that reports of the demise of IBM's midrange DS6000 arrays are not exaggerated. "There has been some demand, but not a large amount," he said. "We don't have big expectations." He added that the XIV Ltd. product would likely replace the DS6000 as IBM's preferred open system in high, midrange and lower enterprise shops.
Finally, Nextra positioning that makes some sense - a replacement for the MIDRANGE OPEN SYSTEMS space previously occupied (poorly) by the DS6000.
Fitting for an architecture that has a higher probability of losing data due to a dual drive failure than does RAID 5. And being built on 7200 RPM SATA drives, with a BEST CASE random read miss response time of about 12 milliseconds, it's clearly not a viable solution for cache-hostile, large block, high performance applications. Or mainframe workloads, for that matter.
don't let me be the last to know
Speaking of high performance applications, Andy did drop some choice tidbits about IBM's focus on flash.
No, he didn't say "Tape" again (and I'm sure there are many IBMers who are very thankful for that).
No, what he did was offer an excuse for why IBM hasn't been updating its premier storage platform (the DS8000) on a regular basis (again, emphasis mine):
Monshaw gave some details on IBM's plans [around flash SSDs] during his talk. He said "the biggest buzz in the industry now" is around SSDs. But first, "systems need to be redesigned," he added. "It will happen in high-performance computing first, then I/O-intense applications in financial services. It's coming into the mainstream. However, latency in the system will not let you take advantage of all its capabilities yet."
Seems that Andy is confirming what I've suspected all along: that the DS8000 can't get much, if ANY, benefit from Flash Drives. The two-node clustered architecture simply can't handle it, and unlike what EMC was able to do with the existing DMX architecture, IBM needs a redesign. There's simply too much latency and overhead, and not enough write cache, to be a viable player in high-performance storage arena.
Truth is, it is very possible to leverage a tremendous amount of performance out of Flash Drives without massively overhauling your architecture.
If you have a strong and versatile architecture to work with.
And Symmetrix DMX most certainly does.
That's why it is able to deliver at least an order of magnitude faster response times (consistently) and up to 30x more IOPS than 15k rpm hard disk drives. Today. Without a massive redesign - just a important few tweaks and tuning here and there. It's times like this that having a robust and flexible architecture really pays off.
And so, after spending a couple of years working on Flash technology, it turns out that the Symmetrix DMX architecture is a rather ideal complement to the inherent idiosyncrasies of flash SSDs. For example, true dynamic cache partitioning (instead of only 2, hard-coded, 50/50 LPARs) allows for precise cache tuning to match the exact write pending throughput of the drives. With this set appropriately, the cache fast-write buffering and write coalescence that Symmetrix does has a HUGE benefit to the expected lifetime of the drive while not unnecessarily wasting expensive SDRAM for buffering reads. And this can be translated into not only improved lifetime for flash drives, but also better overall system performance, without penalizing the hard drive-based applications.
Architectures that can't dynamically partition cache resources or effectively buffer writes for more than a few milliseconds will by definition wear flash drives out faster - and I'm specifically talking about architectures like the DS8000 (with it's infinitesimal write cache) and the SVC (which essentially avoids buffering altogether in order to minimize the inherent overhead it adds to every single I/O - a costly trade-off for flash write endurance).
Without write caching, there can be no doubt that you'd be playing Russian Roulette with your data if you stored on most any of today's IBM architectures. Especially using the SVC, since it has no end-to end DIF checking to ensure that the drive actually returned the bits as they were previously written. You may recall that the smart folks at CERN were surprised to learn just how pervasive such silent bit errors can be with disk drives -- and the situation could be much worse with unverified flash I/O.
Yup - architectural changes are required over in the land of Big Blue, to be sure.
Time to start busting some of the FUD...
Early testers of Flash Drives in DMX are learning that the system is able to take advantage TODAY of far more of the IOPS and Response Time potential offered up by the STEC drives than I'm sure Barry Whyte would like you to believe. He's been harping on the rated 512-byte IOPS of the STEC drive, and he's even asserted that he has validated these claims. And he's trying to imply that the Symm can't take advantage of all the IOPS in the drive.
What he hasn't yet admitted is that those numbers are "specs" and not real-world expectations for the drive. He knows full-well that the drives won't deliver those sorts of Read *or* Write IOPS under normal random I/O workloads...in fact, you can get those Read IOPS only by doing 100% Reads of 512 byte blocks for an extended time - long enough for the drive to adapt its internal tuning parameters to favor reads 100%. Same for writes - the drive can self-optimize to maximize IOPS based on "pure" read or write workloads.
Admittedly, that's great for the spec wars. But meaningless in the real world, where there are ALWAYS reads mixed with writes, and practically always dynamically changing the bias to one side vs. the other over time.
More significant is the fact that the vast majority of applications, file systems, and databases never do ANY 512-byte I/O's! For example, Oracle and Exchange 2007 always do 8Kb reads and writes. In fact, I have seen traces of Symmetrix systems where 40% of the I/O's are 8K - and the rest are larger than that!
And guess what - no flash drive that you can buy today can do 50,000 8K random I/O's per second. NONE of them. And I seriously doubt that there will be any that even approach that rate in the next 2-3 years, either.
Add to that the fact that the faster you write to the drive, the faster it will wear out. And, the smaller the block size, the more writes are "amplified" by the drive to cause writes to an area larger than 512 bytes. Meaning small writes will wear out a flash drive faster than big ones will. In fact, if Sir Whyte were to have his testbed randomly write 512 byte blocks as fast as possible to the drive (in its default configuration), he'd probably be surprised by how fast it wears out.
But then, he's probably not all that concerned with reliability - he's in performance engineering, after all.
Oh - wanna know what the optimal random write size for the ZeusIOPS drive is?
You guessed it: 8 Kilobytes - exactly.
(Conveniently aligned to the DMX architecture, by the way).
i can't make you love me
(Not that I'm trying, that is. It's just a song title, and I needed a transition.)
But, one last point on performance. Andy mentions flash as first benefiting "high performance computing." To that claim, I advise caution - reducing the read miss I/O response time (is the primary benefit of Flash Drives) does NOT necessarily mean your application will run faster - even if your application has a high read miss component. If the application is multi-threaded and isn't pending (idle) waiting on the I/O, speeding up the I/O won't make much of a difference. Or, if your server CPU is fully utilized (like, say, you might be when running multiple instances on a server with VMware) - speeding up the I/O probably won't help the net throughput of the server - in fact, it could make OTHER applications on that server run slower.
So yes - what early adopters are learning is that they may in fact have to re-architect...
...their applications, that is. Or at least, they may have to re-think their server consolidation strategies.
Because believe it, or not, a LOT of server consolidation has actually been enabled by the fact that applications often have to wait on I/O. It's during those wait times that the processor complex can be "time-shared" by VMware (et al).
Take away the wait, and you probably can run fewer applications on the same hardware.
At least, if you want to take advantage of the faster I/O.
Just something you should be considering, so you're not surprised if your application doesn't seem to go faster when you introduce flash - you really have to make sure your infrastructure can take advantage of the performance that Flash can deliver.
I'd like to think that you are the lucky one; if you stuck through this post to the end, perhaps you learned a little more about the realities of Flash performance, rated specs and FUD vs. realities, and some considerations you may have to factor in should you decide to move to flash.
And if you're already a DMX-4 customer, you're even luckier - because you can start taking advantage of this new technology long before IBM customers will get the chance. If you're not sure whether Flash Drives would benefit your application, have a chat with your local Symmetrix SPEED Guru - they can help you identify the performance signatures that indicate whether read miss response times are holding back your application.
I'll bet you're probably surprised at the opportunities.
Better yet - why don't you call up your EMC sales rep and see if they can get you a few drives in for an evaluation? I'm not saying they can: availability is limited, and you may have to put up some collateral on the loan. But there's no better way to know how much you can gain than trying them yourself.
And the sooner you get in line, the sooner you start leaving your competition in the dust.
don't go knockin' on my door
Until we get to EMC World, that is...see you there!
DISCLAIMER: the storage anarchist is most definitely not a Britney Spears fan. In fact, I don't think I have ever intentionally listened to any of her music until today, and then only to link the MP3 file. I'll say nothing pro nor con - I'm just not a fan. Of course, that could change if my subscription base goes up by 25% or more over the weekend.