Support for open-channel SSDs and flash-agnostic APIs

This proposal has been accepted as a session.

*
Accepted Session
File and Storage Systems
Microconference Session

One Line Summary

Integration of flash-agnostic APIs (e.g., Key-value store, Object store, Atomic-IOs, etc.) with open-channel SSDs- status and discussion

Abstract

SSDs with intelligent host-side device drivers have been available for many years (e.g., FusionIO, Virident, Violin Memory). These device drivers offload placement of data to the host system. By moving flash management logic to the host, the design space for alternative storage interfaces that go beyond the traditional block device interface opens up. Examples of these interfaces include flash-optimized file-systems, key-value stores, and atomic IO accesses. Furthermore, block allocation and data placement policies could become pluggable, allowing end-users to select the best strategy for their workload.

Host-side SSD management is a great advantage for vendors implementing their own specialized APIs. However, it is not possible to leverage the alternative storage interfaces we mentioned before in a generic way. By having an open implementation that can be used by multiple SSD vendors, a generic, well-maintained infrastructure can be built. This infrastructure is paired with a new type of open-channel SSDs. These devices expose specific “open-channel” features, such as direct access to read/write/erase of flash pages and discovery of device configuration. Such an infrastructure enables: (I) higher performance than block-based SSDs, (II) more predictability in regards to latency and throughput, and (III) managing multiple SSDs (i.e. their flash) as a single pool.

In this context, we develop LightNVM, a host-side driver for open-channel SSDs. It implements logic, such as flash block management, data placement, bad block management, etc., for a single- or a pool of SSDs. It currently integrates with device drivers that support blk-mq and enables management of them from the host-side.The goal with LightNVM is to create a standard for host-managed SSDs that will make it easy to plug into the Linux ecosystem, and to provide an implementation for it.

We would like to discuss current and future design decisions and how the key-value/object-store/atomic IO/etc. APIs can be integrated into file-systems, such as Ceph, GlusterFS, F2FS; as well as database systems, such as MySQL-based, LevelDB, and others.

RFC patch has been posted here:
https://lkml.org/lkml/2014/10/8/414

Tags

kernel, SSD, open-channel, ceph, glusterfs, f2fs, dbms

Presentation Materials

slides
Back to list of sessions

Speakers