Don’t rewrite your driver. 80 storage drivers for containers rolled into one!

(updated on )

Do you work with containers but your storage doesn’t support your Container Orchestration system?
Have you or your company already developed an Openstack/Cinder storage driver and now you have to do it again for containers?
Are you having trouble deciding how to balance your engineering force between storage driver development in OpenStack, Containers, Ansible, etc?
Then read on, as your life may be about to get better.

Introduction

For a long time, each Container Orchestration solution had its own storage drivers. This burdens vendors with the development of one-off drivers for each Orchestration solution, or pick one and ignore the others.

In early 2017, the wg-storage CNCF working group was formed to explore cloud native storage technology and concepts. Their initial project was the Container Storage Interface (CSI) that aims to define an industry standard storage interface so storage vendors could write a driver once and have it work across a number of container orchestration systems.

Vendors and users will benefit from having a standard. This standard enables user choice, simplifies usage, and adds more value and relevance to particular knowledge, etc.

It’s a year later and the CSI specification is looking good. It’s a living spec that is constantly evolving to improve and adapt to real-world requirements of the container world. The spec is deservedly gaining more and more traction as vendors ramp up providing plugins that support it.

Drawback

The CSI specification is still in the drafting phase, so there’s no stable version that vendors can just implement to consider the work “finished”.

Just like in any other specification, being in the drafting phase means that backward compatibility between versions are not even given priority. The clear priority is to create the best possible specification that meets requirements while accommodating the wider range of storage solutions and usage scenarios possible.

Drivers require constant modifications to stay up-to-date with the specs. Having implementations that stay current and can validate the decisions taken and provide feedback greatly benefits the drafting process.

In the end, this increases the burden on the developers, as they have to implement and reimplement their drivers.

Cinder

Cinder is the Python block storage service within OpenStack that provides volumes for Nova, the compute service. Cinder can create empty volumes, cloned volumes, volumes from images, snapshots, backups, migrate volumes between backends, and much more. All these features are exposed by the service using a common REST API that abstracts storage differences, yet allows particular storage features, like deduplication and compression, to be exposed and used.

The number of supported features is extensive, but they are unlikely to surprise you since they are expected for any mature storage system. What’s really impressive is the ecosystem of storage drivers that are supported by Cinder. No less than 80 drivers are listed in the project’s drivers pages with some well known names such as:

  • CoprHD.
  • DataCore SANsymphony.
  • Dell-EMC: PS Series, Unity, VNX, Storage Center.
  • ScaleIO.
  • VMAX.
  • XtremIO.
  • DISCO.
  • DRBD.
  • Fujitsu ETERNUS DX S3.
  • Huawei FusionStorage.
  • HGST.
  • HPE: LeftHand, 3PAR, MSA, Nimble.
  • Huawei OceanStor.
  • IBM: FlashSystem, XIV, Spectrum Accelerate, FlashSystem A9000, FlashSystem A9000R and DS8000, Storwize V7000, and SVC.
  • GPFS.
  • INFINIDAT InfiniBox.
  • INSPUR InStorage.
  • Kaminario.
  • Lenovo.
  • M-Series Snapshot.
  • NetApp C-mode, E-Series.
  • Nexenta.
  • Prophetstor DPL.
  • Pure.
  • QNAP.
  • Quobyte USP.
  • RBD/Ceph.
  • Sheepdog.
  • Solidfire.
  • StorPool.
  • Synology.
  • Tintri.
  • HyperScale.
  • ACCESS Share.
  • Veritas Clustered NFS.
  • VMware VStorageObject and vCenter.
  • Virtuozzo.
  • Windows Storage and SMBFS.
  • Zadara VPSA.
  • ZFSSA.

In the Cinder context, “supported” means drivers with active developers and their own CI system to validate every single patch submitted to both the Cinder project and the driver itself against real hardware.

cinderlib-csi

On one hand we have the CSI specifications, an in-progress draft with a great future in the container world, and on the other we have the Cinder drivers, feature rich and field tested storage drivers meant for VMs. What if we could fuse them to provide the ultimate CSI driver? One that could bring all the Cinder drivers to the container world.

It was not until the recent appearance of the cinderlib library that Cinder drivers could be used without running the Cinder service. Thanks to this library it is now possible to create our own applications that use the drivers’ Python code to manage supported storage solutions.

And this is where the cinderlib-csi driver comes in. It provides a storage backend agnostic CSI driver for all the Cinder drivers, providing block volumes to containers. We are no longer talking about a CSI driver that interfaces via REST API to a standalone or full blown Cinder service. It means not having a DB, a message broker, or even running any of the Cinder services. This greatly simplifies deployment and maintenance, not to mention the memory, CPU, and memory savings.

The cinderlib-csi driver is a proof of concept, but it’s the most up to date CSI driver, supporting not only v0.2.0, but also last week’s snapshot features as well for a great number of storage backends and connection types!

Operations supported by the driver are:

  • Creating block volumes.
  • Creating snapshots.
  • Creating a block volume from a snapshot.
  • Deleting block volumes.
  • Deleting snapshots.
  • Listing volumes with pagination.
  • Listing snapshots with pagination.
  • Attaching volumes.
  • Detaching volumes.
  • Reporting storage capacity.
  • Probing nodes.
  • Retrieving plugin information with all version information.

For now, we need a database to store the metadata, but the plan is to use the cinderlib metadata plugin to create specific plugins to leverage the container orchestration features. For example, for Kubernetes we would use a CRD plugin that would store the metadata directly in Kubernetes as CRDs, thus removing the need to deploy a DBMS.

Testing the cinderlib-csi plugin can be done on baremetal and in a container. Deploying a DBMS is not necessary, we can use a SQLite database as it’s done in the repository examples.

What’s next

The plugin is still in the initial development phase and must be considered a proof of concept. In this stage, there are many things that need to be done. The priority will be split between adding new features and improving the robustness.

Some of the things that we have in mind are:

  • Supporting filesystem volumes
  • Supporting NFS Cinder backends
  • Creating the Kubernetes CRD metadata plugin
  • Increase the checking of the received parameters

If you found interesting the use of cinderlib, you may also want to have a look at how it can be leveraged in Ansible and how it can be used in your own applications.

Feedback is always welcome, on github, by email, or on IRC (my handle is geguileo and I hang on the #openstack-cinder channel in Freenode).