FLASH: There’s danger in them waters.

By Juan Orlandini
9/20/2011

FLASH is fast!

That's the banner everyone's been waving for a while. We've all bought into it. I sure have. My shiny new MacBook Air with SSD feels much faster than anything I've ever had. As my teenager says, "it's crazy fast." Applications load essentially instantly. The same has to be true for the fancier enterprise SSDs, right? Spinning disk is dead! Dead I tell you!

Or is it?

Early last year, a colleague asked me to predict when spinning disk would fade out. At the time, I was fully sucked into the hype of FLASH, and predicted that in three years it would all be FLASH. I take it back. Here's my new take: Spinning disk is going to do what tape did – it'll stick around for a long time. When disk-based deduplication for backups were introduced, everyone predicted tape would die. It didn't. It still has a place. The economics are simple. For long-term retention, you likely need tape. Disk will go through a similar track. So how's that compare? Let's look at what FLASH is good for:

  • Lots of IOPS (for some workloads)
  • Low power per IOP (for some workloads)

What does "for some workloads" mean? Turns out that there are places where the advantages of FLASH are simply not true. If your IOPS are large sequential reads or writes, wide stripes of spinning disks are going to flat out crush FLASH, especially if you look at the economics. If you need a gazillion IOPS (and yes, gazillion is a technical term), you may not be able to take advantage of FLASH or even have FLASH meet your requirements. Traditional array vendors have optimized their architectures for spinning disks. They have controllers that do all sorts of magic to minimize the disadvantages of spinning disk. Cool things like RAID, caching, multiple paths, virtualization (lots of disk as one = easy to administer), etc. In a large sense, you actually don't need this stuff for FLASH. But if you put FLASH behind these controllers, you still have all that code, all that magic. But, it's in the way of getting to the FLASH. If price wasn't an option, you'd never buy an HDS VSP or EMC DMX with all FLASH drives. The controllers would never satisfy the IOPS those drives can handle. We're seeing a ton of vendors that are developing FLASH-optimized storage solutions (TMS, Virident, Fusion-io, Intel, OCZ, etc.). Compared to spinning disk, all these guys are really, really fast.

You mean there's something other than disk to compare to?
Yup. Your servers' fastest I/O isn't coming from secondary storage; it's coming from DRAM. Well, actually, it's coming from the cache on your CPUs, but that's a topic for another time. Companies that are looking for the fastest I/O are not going to FLASH. They're using servers with TONS of DRAM. As you know, you can buy servers with 1TB of RAM or more now days. It's expensive, but it's really, really fast. FLASH doesn't even come close. For extreme performance, you have to go down this path. But, what if your I/O requirements are huge (really huge), and you can't buy or fit enough DRAM into your server? That's when it gets interesting.

FLASH is only so fast.

To keep these truly high I/O systems operating at peak performance, you have to start using FLASH as a cache technology, an HSM technology, or both for your DRAM. To do so, you have to come close to matching the speed of DRAM or you'll need more than one FLASH storage solution. With DRAM, your I/Os (if you call them that) are measured in the microseconds. Doing some fancy math (ask me if you're curious), you can easily determine that in DRAM you can do millions of I/Os per second. Millions. The fastest FLASH arrays can sustain around 25 to 30K IOPS per device (let's not get into FLASH degradation, rewrite penalties, all that other stuff – that's another blog). For truly large I/O requirements, you'll need multiple FLASH devices.

So you buy lots of FLASH devices. What's the big deal, other than money? Well, you're now managing lots of discrete devices. Remember the cool stuff that big arrays do for you? Remember the one where they manage lots of devices and make it look like one so it's easy for you? The one that makes lots of devices redundant? All that stuff that makes it enterprise class? Without this magic, you have to manage the FLASH stuff yourself. Guess what the most expensive part of this is. The people, process stuff. The stuff we're supposed to be making easier and cheaper. Doh!

FLASH is very cool. It's way faster than what we're used to, but it's not über fast. You still need to understand where it fits, what its downsides are (I didn't get into that – yet), and that you're using it at the right place. You even have to know the right vendor for the application you're trying to solve. But if you're looking for the fastest, you need to be careful.

As usual, measure twice. Cut once. Use spinning disk, FLASH, and even DRAM where appropriate.