Put a small cache on a system and the very hottest data on that system, the half a percent that gets 50% or more of the I/Os doing metadata, or database index lookups.Īs you increase the size of the cache it has to hold more data, but that new data is always less active, and soon you’ve hit the land of diminishing returns where you need 10X as much cache to get any significant increase in the cache hit rate. You would think that you could just make the cache bigger, to keep the number of cache misses down, but that ignores the fact that cache effectiveness falls off dramatically with cache size. Now requests back up in queues and latency climbs dramatically from 5-20ms to hundreds of ms, this causes user’s video players to stutter, and even worse, to re-try as they time out creating even more I/Os, which queue up and create more latency. The problem comes on the weekend our video service releases the hotly expected new movie, thousands of new viewers arrive, and the number of cache misses exceeds the small number of IOPS a back end of 20 or 40 TB hard drives can handle. The system may be able to satisfy 90% of I/Os from a cache of a few SSDs under so called normal conditions even then users will report inconsistent performance as the latency of each cache miss is the 5-10ms it takes to read from disk 20-40 times the 500µs SSD read latency.
You would think that with big hard drives delivering so few IOPS/TB caching the hot data and metadata is a good idea…and it is, but a cache can’t completely make up for a lack of IOPS in it’s backing store, especially when facing demand peaks. We also didn’t include in our discussion of slot costs the cost of any SSDs for cache, or metadata store, and who runs a pure HDD storage system today?
#QUICK DISK DIMENSIONS SOFTWARE#
Put commercial storage software like VSAN (~$11,000/node) on that server and your cost per slot goes up another $180 (with 60 drives per server) to $900 with 12 HDDs/server. The $200-$800 per slot costs included in the table, are low, even for a roll-your-own solution covering basically just the server the drives will plug into. Move to the 1U server with 12 HDDs at $500/slot and a system with 18TB drives costs 1/3 rd of what the same system with 4TB drives would. For the lowest cost option, a 90 HDD 4U server from SuperMicro and open source software like Ceph at ~$200/slot, using 4TB hard drives will cost you twice as much as using 12 TB or bigger drives. Things become very different when we add the cost of the slot in your storage system that hard drive plugs into. Compare to Scale-up file system, the software and feature licenses were three times the cost of the hardware reducing the cost of disk drives to less than a quarter of storage system costs.Īs you can see in the table below, hard drives have a pretty consistent cost per TB from 4-12TB with up to about a 50% premium for the very biggest drives. Scale-Out storage systems can usually scale larger than systems based on the older scale-up architecture. When we priced out a leading scale-out Scale-Out storage systems allow users to grow a system by adding compute power along with media in the form of HDDs and SSDs.
#QUICK DISK DIMENSIONS FREE#
The per slot cost of software covers a much bigger territory from essentially free open source software to the software licenses for enterprise storage which can be several times the cost of the hardware. Enterprise storage hardware usually ends up costing $500-$1,000 a slot. At the low end, a SuperMicro 1U server holding 12 3.5” disks, like you might use for an object store, sells for $5,600 or $466/slot, their 4U 90 HDD behemoth with redundant servers brings that down to $180/slot. You can buy a 60 drive SAS JBOD for about $6,000 or $100/slot but that JBOD needs to be connected to an existing storage system or server. To build a storage system, you’ll need hardware and software. Today, you’ll pay more for the JBODs, servers, software and support than for spinning disk drives, what we call the slot costs. It’s important to remember that it’s been a long time since the vast majority of the cost of a disk array was the disk drives. At that level, even applications we think of as sequential, like video servers, start to stress the hard drives.Ī video-on-demand system doesn’t serve one user a single sequential video stream it serves 1,000s of users 1,000s of streams and the hard drives have to move their heads back and forth between Frozen, The Mandalorian and the thousands of other videos customers are watching.
As disk drives grow over 10 TB, the I/O density falls to just a handful of IOPS per TB.
0 kommentar(er)
