shuffle sharding

Volumes are spread somewhat-randomly throughout a cell, and each OSD holds several thousand volumes. This means that if we lose a single OSD we can reconstruct the full set of volumes from hundreds of other OSDs simultaneously. This allows us to amortize the reconstruction traffic across hundreds of network cards and thousands of disk spindles to minimize recovery time. » about 300 words

An incomplete lit review on balancing data and request volumes between shards for performance and reliability, with an extra bit about replicating requests to maximize both. » about 1000 words