Introduction to VMware vSAN Stretched Cluster
VMware vSAN Stretched Cluster was introduced back in vSAN 6.1 specific configuration for the environments where disaster / downtime is a primary requirement. Stretched Cluster deployment will be having 2 Active / Active Data Site connected with the well-connected network with a round trip time (RTT) latency not more then 5ms. Both the data sites will be connected to third site having vSAN Witness host to avoid “Split-Brian” issue if in case connectivity is lost between the data sites. You can have a maximum of 31 host in a vSAN Stretched Cluster deployment (15 host in each data site and 1 Witness host in the third site). Virtual machine deployed in a vSAN Stretched Cluster will have one copy of data in data site and witness components will be in third site.
vSAN Stretched Cluster implementation limitation
- In a vSAN Stretched Clusters, there are only 3 Fault Domains. The maximum number of FTT supported is 1 in pre-vSAN 6.6 stretched cluster.
- SMP-FT is not supported if FT Primary VM and Secondary VM are not running in same location. SMP-FT is supported if Primary & Secondary VM are running in the same fault domain.
- Erasure Coding feature is not supported in Stretched Cluster Configurations pre vSAN 6.6 but is supported for Local Protection within a site when using vSAN stretched cluster and Per-Site Policies
- vSAN iSCSI Target Service is not supported
Networking and Latency Requirements
- VMware recommends vSAN communication between the data sites be over stretched L2.
- VMware recommends vSAN communication between the data sites and the witness site is routed over L3.
- VMware recommends latency of less than or equal to 200 milliseconds in vSAN Stretched Cluster configurations up to 10+10+1. For configurations that are greater than 10+10+1, supported latency of less than or equal to 100 milliseconds.
- Bandwidth between data sites and the witness nodes are dependent on the number of objects residing on vSAN. A standard rule of thumb is 2Mbps for every 1000 components on vSAN.
Bandwidth Calculation
Between Data Sites
Bandwidth requirement between the two data sites is dependent on workload like number of write operations per ESXi host, rebuild traffic need to be factored in. Assuming read locality and there will be no inter-site read traffic, read operations are not required to be factored.
The required bandwidth between the two data sites (B) is equal to the Write bandwidth (Wb) * data multiplier (md) * resynchronization multiplier (mr):
B = Wb *md * mr
VMware recommends to have Data Multiplier value as 1.4. For resynchronization traffic, VMware recommends an additional 25% i.e. mr = 1.25. Assuming 20,000 write IOPS a “typical” 4KB size write would require 40MB/s, or 640Mbps bandwidth.
Bandwidth = 640 * 1.4 * 1.25 = 1120 Mbps
Between Data and Witness Site
The required bandwidth between the Witness and each site is equal to ~1138 B x Number of components /5s. Assuming 1000 components in data site, the bandwidth required between the Witness and Data Site will be
1138 * 8* 1000/5 = 1,820,800 i.e. 1.82 Mbps
With 10% buffer as a rule of thumb we can consider 2 Mbs of bandwidth is required for every 1000 components.
Configuring VMware vSAN Stretched Cluster
Hope this will be informative for you. Thanks for reading. Please share if you find worth sharing it !!!