Storage Class Memory Under the Covers

In a previous blog post I wrote about the introduction of a very exciting class of new storage technology referred to as Storage Class Memory. These technologies can be used in one of 2 modes. It can be configured as regular volatile memory. When used in this way, it’s slightly slower than DRAM, more dense, and presumably cheaper once in the mass market. The other option is for it to be configured as persistent memory. Both flavors are useful and are controlled by the BIOS. Intel/Micron made a bit of splash recently by cryptically announcing a version of this technology called “3D XPoint™.”

Intel/Micron 3D XPoint

Intel/Micron 3D XPoint

In order for memory to be persistent, a few things have to happen. First, the persistence is under program control and contents must be flushed through the processor caches. This is done via one of 3 operations added to the Intel x86 instruction set: CLFlush is the oldest, most heavyweight instruction and it serializes access. CLFLushOPT is newer, has address ranges and many can be issued in parallel. Finally, there is CLWB which performs the same flush to the memory controller, but keeps the content in cache for subsequent lookups. Once the content is in the memory controller via these instructions, it still may not be on the actual persistent memory. For that another instruction is needed, in this case called PCommit.

Microsoft showed off 2 modes of access that they will be delivering in a future version of Windows Server. The 2 modes (block mode and DAS mode) are determined at format time. Block mode is designed for compatibility with existing applications and makes use of NTFS and the full Windows storage stack by adding SCM drivers at the base of the driver stack. It utilizes an Intel software capability called Block Translation Table (BTT) that makes disk block writes atomic on this medium and provides the full Windows storage stack and capabilities to unmodified applications. The native option, Direct Access (DAS) mode, is for new applications being developed for persistent memory. DAS presents byte addressable memory mapped files to applications that are backed by the SCM. In this mode, applications access the persistent memory just like memory mapped files except they need to address the persistence directly. This mode of access bypasses most of the Windows storage stack including NTFS and all filter drivers. The memory mapped files on SCM are integrated with the NT cache manager/memory management subsystem.

Native SCM Storage Stack

Native SCM Storage Stack

Hyper-V support was not disclosed and VMware didn’t speak about their plans at the SNIA Storage Developers Conference. I would expect hypervisors to treat SCM as a shared resource; virtualizing it and exposing specific SCM regions directly to a guest OS VM with the requisite isolation from any other VMs. I would also expect the hypervisor itself to make use of SCM as a system resource usable in its own functional enhancements, such as within the hypervisor IO and storage stack. SCM represents a really innovative and revolutionary new technology for the industry and I for one can’t wait to start working with it at Infinio!

This entry was posted in Evolution of Storage Architecture. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *