Overview
An endangered chunk is data that would become inaccessible if any single storage node went offline. XOR protection is one of the EFS redundancy schemes that prevents this, by storing a parity part alongside each chunk's data parts.
EFS splits every file into chunks, up to 2 GiB each, and spreads them across chunk servers (storage nodes). How safe a chunk is depends on its storage goal, which sets the redundancy scheme and where the parts may reside. EFS offers three schemes:
| Scheme | How it protects data | Survives |
|---|---|---|
No redundancy (e.g. 1, 1@hdd) |
One copy on one server; relies on hardware RAID only | No node loss |
Duplication (e.g. 2@ssd) |
N full copies on different nodes | Up to N-1 nodes offline |
XOR protection (e.g. xor3@_) |
N data parts + 1 parity part, each on a different node | 1 node offline |
In practice, XOR is used on systems with at least three storage nodes that also have hardware RAID.
How XOR protection works
XOR protection stripes each chunk into N equal data parts and computes one parity part of the same size. Each part goes to a different storage node. If one node drops out, EFS rebuilds the missing part from the survivors.
For example, on a 5-node system a 1 GB chunk can be stored as xor4: four 250 MB data parts plus one 250 MB parity part, 1.25 GB in total. That is 25% overhead, compared with 100% for two full copies.
The trade-off is that XOR has exactly one parity part. The data stays readable and writable with one node offline, but not two. This holds only because EFS normally never places two parts of the same chunk on one server.
Disk Space used when writing a 2GB File
| Storage Goal | Disk Space Used | Minimum Storage Nodes |
|---|---|---|
| Copy1 | 2 GB | 1 |
| Copy2 | 4 GB | 2 |
| XOR2 | 3 GB | 3 |
| XOR3 | 2.67 GB | 4 |
| XOR4 | 2.5 GB | 5 |
What an endangered chunk is
An endangered chunk is one that may become unavailable for reading and writing if any storage node in the EFS system goes offline. The EFS Status Page reports the count in the Statistics of chunks table on the Info pane.
Whether that count is a problem depends on the system:
| Situation | Endangered chunks are... |
|---|---|
| Single-node EFS system | Expected for every chunk; there is nowhere else to put redundant parts |
Chunks in a no-redundancy goal (1, 1@_, 1@SSD) |
Expected; a single copy always depends on one node |
| Chunks in a redundant goal (duplication, or XOR) on a multi-node system | Not expected; a part or copy is missing and the goal is not met |
In the last case, an XOR chunk has already lost its one spare part, usually because a node is down or hasn't finished registering. It now has zero tolerance: losing one more node makes it unreadable. EFS treats these chunks as undergoal, and they show up as chunks with missing parts.
If you have any questions, please reach out to EditShare Support.
Comments
0 comments
Article is closed for comments.