At the SNIA Storage Developers Conference last week, I immersed myself in what is potentially the most disruptive storage technology innovation that I’ve seen during my career. Yes, even more disruptive than flash! This technology will likely take a couple of years to mature and become mainstream, but when it does, watch out! You heard it here first. I am referring to Storage Class Memory (SCM) aka Persistent Memory. This technology represents a new hybrid storage/memory tier with unique characteristics that I predict will change the way we build applications, how they access their storage, and how we build storage devices themselves.
SCM is not memory and it’s not storage. It’s actually something unique. These new SCM devices connect to memory slots in a computer and they are mapped and accessed similarly to memory although they are slightly slower. Unlike previous generations of storage technology, they can be addressed atomically at either the byte level or block level. Operating systems will likely expose them as either very fast block storage devices formatted by file systems and databases (for compatibility) or as direct memory mapped “files” for next generation applications. Hypervisors will likely expose them as new, specially named and isolated SCM regions. They can be used directly as execution memory or data storage memory.
One of the revolutionary aspects is that by declaring SCM as persistent in the BIOS and then executing specific control instructions when accessing it, SCM persists content across crashes and reboots. Even better, there are new atomicity mechanics built into these instructions. And it’s byte level addressable! Compare this to the traditional paradigm of converting data structures into constructs that fit into a 512-byte block and then carefully writing the block and associated meta-data to disk.
To make matters even more interesting, at the same time, there is a renewed look at RDMA technologies for these persistent memories. DMA or Direct Memory Access is used on individual computers, typically to make storage devices or network adapters faster by enabling an IO device to move data directly between memory and a peripheral device without CPU involvement or extra copies. There have been proposals in the past to create various forms of remote DMA over the network, but they have never seen more than niche adoption (sorry Infiniband; the closest predecessor technology that has not achieved ubiquity). However, it appears to me that persistent memory may be the impetus for revisiting RDMA and driving it as a mainstream, ubiquitous technology. There are currently two leading candidates for generic RDMA approaches – RDMA over Converged Ethernet (RoCE) and Internet Wide Area RDMA Prototcol (iWARP).
What is SCM’s real potential impact? Well, since I started computer programming and operating system design during the mini-computer era, file systems, databases and, more recently, key/value pair stores all operated on data in volatile persistent memory. Subsequently, at specific execution points, important data structures were reformatted and placed into 512-byte blocks, then written to disk for persistence. And they had to store enough metadata and be stored with careful ordering to be able to recognize and recover from failures and corruptions. Until now, we have never had the ability to directly persist a live memory data structure at such granularity and without making such an intermediate copy. These new persistent memories enable all this and at speeds that are an order of magnitude faster than flash!.
So, we have an ideal storage technology. It can be addressed at the byte level. It has zero copy persistence. Atomicity is more fine grained and under direct application control. And to withstand server failures, it will likely have network memory mirroring between servers. And since SCM operates at close to server memory speeds, application co-located server-side storage software processing will be imperative for delivering on this technology’s IO performance potential. Who said storage was stale and boring?
In subsequent blogs, I’ll delve into the details of how Storage Class Memory technology works and project how I think it will be used.