So we've talked a lot on the channel about CephFS, why we love it here at 45 Drives, as well as we've gone in a little deeper on how to expand a storage cluster. Now today though we're gonna be talking about what Ceph is at its core, and we're also going to talk about the components that make up a Ceph storage cluster.

So to begin, Ceph is a fully open source, distributed data store owned by Red Hat, which is based on object storage that is designed to provide fantastic performance, scalability, and reliability. So Ceph has the ability to interface with object based interfaces, as well as legacy based ones such as file system or block as well. Ceph has an API layer called “Librados” that supports many languages such as C, C++, Java, and Python, and it is the foundation of what Ceph’s different storage interfaces are built upon. So a great feature of Ceph that makes it extremely robust and reliable is that it allows administrators to provide object-based storage systems through things like S3, as well as block devices through what's called RBD or “RADOS Block Devices”, and finally through file system, and Ceph uses a distributed file system called CephFS.

So what does a Ceph cluster consist of exactly? So Ceph has certain requirements to get you started, and those are at least one Ceph monitor, one Ceph manager, and one Ceph OSD. So you'll also require a Ceph metadata server if you plan on running CephFS. So while you really only need one of each of these services to stand up a cluster, there is a pretty big difference between standing up a cluster and then having a production ready cluster. Ceph was really built to have no single point of failure, and so we really recommend to have between two to three of each of these services running at any time and I'm going to go into detail and each of these services here now.

So first let's talk about the Ceph monitors. So what the Ceph monitor does is it maintains a map of the entire cluster, so it has a copy of the OSD map, the monitor map, the manager map, and finally the crush map itself. So these maps are extremely critical to Ceph for the daemons to coordinate with each other. We require least three monitors when building Ceph clusters to ensure high availability, redundancy, and for the monitors to reach quorum.

So next we have Ceph managers. Ceph managers do things like keeping track of the runtime metrics, as well as your system utilization, things like your CPU performance, disk load, things like that. So these Ceph managers also host the Ceph dashboard web GUI, which allows administrators to get a full picture of the cluster health as well as perform most tasks required. So we typically recommend three managers, although two will suffice.

Next is the Ceph OSD’s. So Ceph has something called an OSD or an “Object Storage Daemon”, but it also has things called OSD nodes. So OSD nodes are where the OSD’s live. So with our clusters, the minimum OSD nodes to begin with is 3. So the OSD is where your data is stored, and they also handle things like rebalancing and replication. So OSD’s will also send some information to your monitors and your managers. So in nearly all cases you will have one OSD per HDD or SSD in your cluster. So you'll most likely have dozens or possibly hundreds or thousands of OSD’s depending on how big your cluster is, although three is the minimum to ensure redundancy and high availability.

So next up are the ReDoS gateways. These are object-based interfaces that are built upon the Librados API, and these are meant to provide the end-user with RESTful based object storage into the cluster. So the RGW currently supports two different interfaces and those are S3 and Swift. So one unique feature about this is if the end user wants to use both the S3 and the Swift API; because it's under the same namespace, they can then write under one API and read with the other.

So finally we come to the Ceph metadata servers, so these are only required if you're using Ceph’s distributed file system called CephFS. So they handle the storing of metadata for CephFS and allow for better performance by caching the metadata in RAM to allow for faster access to it. So we typically like to have two metadata servers on our clusters when we're running CephFS, and will typically bind those to our file system gateways.

So that was an overview of what Ceph is as well as a deeper dive into the components that it's made up of. So before we like to go, we usually like to give fun fact. So this isn't so much as a fun fact as much as it is “I forgot to talk about it in the main part of this video”, so here we go.

There's one other part of the Ceph cluster that I haven't talked about and it really is integral to all Ceph clusters, and that's the network. So all these nodes have to communicate somehow, and the way Ceph does it is through networking and a LAN. Now there's a few different ways you could do this but the way we like to do it is to have a private network for your OSD’s and a public network for the rest of the cluster, as well as your gateways and anything that's client-facing. The reason why you do this is because the OSD nodes are really chatty. They handle self-healing, replication, and moving data through the cluster, and so it's best to keep that off the public network if you can, because that gives more bandwidth for IO and things that clients want to do on the cluster. And so Ceph can run on one gigabit networks, but we typically recommend at least having 10 gigabit networks here, and especially if you can swing it, we like to do LACP bonded 10 gigabit networks to give you 20 gigabit of bandwidth over your private networks, as well as your public networks as well if you can handle it.

Alright, well thanks for watching Everyone. Hopefully you found some of this interesting and it gave you a better overview of what Ceph is at its core, and now going forward you'll be able to understand some of the other videos a little better. So if you have any questions or comments, be sure to leave them down below, thanks for watching and we'll see you next week on another tech tip.