Add info/DCtoDC_interconnection_types.txt
This commit is contained in:
279
info/DCtoDC_interconnection_types.txt
Normal file
279
info/DCtoDC_interconnection_types.txt
Normal file
@@ -0,0 +1,279 @@
|
||||
# 🌍 **1. DC ↔ DC Interconnects: The Reality in Practice**
|
||||
|
||||
Datacenters never rely on “one link”.
|
||||
They use **multiple layers of transport**, each chosen depending on distance, risk, latency, and cost.
|
||||
|
||||
We group them into:
|
||||
|
||||
1. **Metro / Intra-city DC-DC links (0–80 km)**
|
||||
2. **Regional DC-DC links (80–600 km)**
|
||||
3. **Nation / Cross-country DC-DC links (600–2000+ km)**
|
||||
4. **International / Cross-continent DC-DC links (2,000–15,000+ km)**
|
||||
|
||||
Each layer uses different **fiber types**, **DWDM systems**, **optical budgets**, and **routing architectures**.
|
||||
|
||||
---
|
||||
|
||||
# 🏢 **2. Metro / Intra-City DC-DC Links (0–80 km)**
|
||||
|
||||
**Use case:** connecting two datacenters in the same city or urban area.
|
||||
|
||||
### ✔ Common Technologies
|
||||
|
||||
**a) Dark Fiber (most common)**
|
||||
|
||||
* Pure unlit fiber pairs leased from a provider or owned.
|
||||
* You bring your own optics (100G, 400G, ZR/ZR+, 800G).
|
||||
* Best for latency-sensitive workloads (AI clusters, storage replication).
|
||||
|
||||
**b) Metro Wave Services (10G/100G/400G wavelength)**
|
||||
|
||||
* Provider manages DWDM transport.
|
||||
* One wavelength = your dedicated bandwidth.
|
||||
|
||||
**c) Short-haul DWDM**
|
||||
|
||||
* Ciena / Infinera / Nokia boxes on your premises.
|
||||
* Up to 80–100 km without amplification.
|
||||
|
||||
### ✔ Typical Link Types
|
||||
|
||||
* **Dual diverse fiber paths** via different ducts and buildings.
|
||||
* **100G LR4 / 400G LR8 / 400G ZR** depending on distance.
|
||||
* **EVPN-VXLAN** overlays for L2/L3 DC fabrics.
|
||||
* **IP/MPLS** or **SR-MPLS/SRv6** for transport.
|
||||
|
||||
### ✔ Metro Latency
|
||||
|
||||
* ~0.45–0.50 ms round-trip per 50 km fiber
|
||||
(25 km one-way ≈ 0.25 ms).
|
||||
|
||||
### ✔ Used By
|
||||
|
||||
AI campuses, Tier-1 banks, cloud providers (AWS/Azure/Google), telecom hotels.
|
||||
|
||||
---
|
||||
|
||||
# 🗺 **3. Regional DC-DC Links (80–600 km)**
|
||||
|
||||
Distances too long for LR optics but perfect for **ZR+ optics** or **amplified DWDM**.
|
||||
|
||||
### ✔ Common Technologies
|
||||
|
||||
**a) Long-Haul DWDM with EDFA + Raman amplification**
|
||||
You own the line system or provider manages it.
|
||||
|
||||
**b) Coherent optics (100G–800G)**
|
||||
|
||||
* QSFP-DD ZR+ (400G up to ~120 km unamplified, 2000 km with DWDM).
|
||||
* 800G modules for <200 km high throughput AI fabrics.
|
||||
|
||||
**c) Provider Wavelength Services**
|
||||
|
||||
* 10G/100G/400G on long-haul DWDM backbone.
|
||||
* Guaranteed SLA.
|
||||
|
||||
**d) MPLS/SRv6 Transport Networks**
|
||||
If you connect multiple regions into one backbone.
|
||||
|
||||
### ✔ Typical Link Types
|
||||
|
||||
* **Dual wave 100G/400G** in active/passive mode
|
||||
* **EVPN-MPLS** or **EVPN-VXLAN over MPLS**
|
||||
* **Dedicated L2 circuits** for synchronous storage replication (low-latency DR).
|
||||
|
||||
### ✔ Latency Examples
|
||||
|
||||
* 200 km ≈ 1 ms RTT
|
||||
* 500 km ≈ 2.5–3 ms RTT
|
||||
|
||||
### ✔ Used By
|
||||
|
||||
* Multi-region platform providers (Oracle, Azure)
|
||||
* Financial institutions
|
||||
* Telco/ISP regional POP interconnects
|
||||
|
||||
---
|
||||
|
||||
# 🇪🇺 **4. Nationwide / Cross-Country DC-DC Links (600–2000+ km)**
|
||||
|
||||
At this distance:
|
||||
|
||||
* Pure dark fiber becomes hard (expensive to lease over entire country).
|
||||
* Most companies use **carrier-provided coherent wavelength services**.
|
||||
* Large players deploy **their own long-haul DWDM networks**.
|
||||
|
||||
### ✔ Transport Technologies
|
||||
|
||||
**a) Ultra Long-Haul DWDM**
|
||||
|
||||
* Ciena 6500 / Infinera DTN-X / Nokia 1830 PSS
|
||||
* Amplifiers every 80–120 km
|
||||
* ROADMs for dynamic circuit switching
|
||||
|
||||
**b) Coherent Optics**
|
||||
100G–800G on QPSK/16QAM modulation
|
||||
|
||||
* Higher modulation = shorter reach
|
||||
* Lower modulation = longer reach
|
||||
|
||||
**c) MPLS / SR-MPLS / SRv6 Backbone**
|
||||
IP layer sits on top of optical layer.
|
||||
|
||||
**d) Encrypted Wave Services**
|
||||
AES-256 hardware encryption for compliance (PCI, GDPR).
|
||||
|
||||
### ✔ Latency Examples
|
||||
|
||||
* Bucharest → Frankfurt (~1,500 km fiber): ~11–14 ms RTT
|
||||
* Amsterdam → Stockholm (~900 km fiber): ~6–7 ms RTT
|
||||
|
||||
### ✔ Common Link Types
|
||||
|
||||
* **100G/400G waves**
|
||||
* **EVPN-MPLS** for DC interconnect
|
||||
* **L3VPN and L2VPN** for isolation
|
||||
|
||||
---
|
||||
|
||||
# 🌍 **5. Intercontinental DC-DC Links (2,000–15,000 km)**
|
||||
|
||||
Now we are at **submarine cable** level.
|
||||
|
||||
### ✔ Technologies
|
||||
|
||||
**a) Submarine Fiber Cables**
|
||||
With repeaters every 60–80 km.
|
||||
|
||||
Top systems today:
|
||||
|
||||
* Google Dunant (France ↔ USA) 250 Tbps
|
||||
* MAREA (Spain ↔ USA) 200 Tbps
|
||||
* SEA-ME-WE 6 (Southeast Asia ↔ Europe)
|
||||
|
||||
**b) Coherent transponders on cable landing stations**
|
||||
100G/200G/400G line rates.
|
||||
|
||||
**c) Provider services**
|
||||
|
||||
* 10G/100G/400G waves
|
||||
* Private line over subsea systems
|
||||
* IP transit or private peering across oceans
|
||||
|
||||
### ✔ Latency Examples
|
||||
|
||||
* Europe ↔ US East: ~70–80 ms RTT
|
||||
* Europe ↔ Singapore: ~150–170 ms RTT
|
||||
* US ↔ Tokyo: ~90–110 ms RTT
|
||||
|
||||
### ✔ Routing Architecture
|
||||
|
||||
* **Global backbone using MPLS/SRv6**
|
||||
* **EVPN for DC-to-DC traffic**
|
||||
* **Segment Routing across continents**
|
||||
* **Multiple path diversity using different ocean cables**
|
||||
|
||||
### ✔ Why not dark fiber?
|
||||
|
||||
You cannot own submarine dark fiber unless you are:
|
||||
|
||||
* Google
|
||||
* Meta
|
||||
* Amazon
|
||||
* Large telcos
|
||||
|
||||
For normal companies: you lease waves.
|
||||
|
||||
---
|
||||
|
||||
# 🧩 **6. DC ↔ DC Link Architecture: How They Are Built**
|
||||
|
||||
### **Every serious datacenter interconnect has:**
|
||||
|
||||
1. **At least 2 physically diverse paths**
|
||||
|
||||
* Different ducts
|
||||
* Different bridges
|
||||
* Different power grids
|
||||
* Different access buildings
|
||||
|
||||
2. **Dual routers per DC**
|
||||
|
||||
* Typically *spine-like*
|
||||
* Multi-chassis LAG or ECMP
|
||||
|
||||
3. **Optical layer redundancy**
|
||||
|
||||
* Two DWDM shelves
|
||||
* Separate line cards, amps, ROADMs
|
||||
|
||||
4. **Traffic engineering**
|
||||
|
||||
* BGP with communities
|
||||
* SR-MPLS / SRv6
|
||||
* Fast reroute (TI-LFA)
|
||||
|
||||
5. **Overlay separation**
|
||||
|
||||
* Underlay: IP or MPLS
|
||||
* Overlay: EVPN-VXLAN
|
||||
|
||||
6. **End-to-end encryption**
|
||||
|
||||
* MACsec (in metro)
|
||||
* Wave encryption (regional)
|
||||
* IPsec where needed
|
||||
|
||||
---
|
||||
|
||||
# 🧱 **7. Common Inter-DC Topologies**
|
||||
|
||||
### ✔ Metro 2-DC Active/Active
|
||||
|
||||
```
|
||||
DC-A ====100G==== DC-B
|
||||
\ /
|
||||
\----100G-----/
|
||||
```
|
||||
|
||||
### ✔ Regional Triangle
|
||||
|
||||
```
|
||||
DC-B
|
||||
/ \
|
||||
100G/ \100G
|
||||
/ \
|
||||
DC-A-------DC-C
|
||||
100G
|
||||
```
|
||||
|
||||
### ✔ International Mesh
|
||||
|
||||
```
|
||||
London ----- Amsterdam
|
||||
\ /
|
||||
\ /
|
||||
Frankfurt
|
||||
/ \
|
||||
Paris ------- Milan
|
||||
```
|
||||
|
||||
### ✔ Global Backbone Loop (Cloud Providers)
|
||||
|
||||
```
|
||||
US-East → Europe → Asia → US-West → US-East
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 🧠 **8. Summary: What Determines the Link Type?**
|
||||
|
||||
| Distance | Optical Layer | Typical Bandwidth | Latency | Cost | Common Use |
|
||||
| ------------- | --------------- | ----------------- | --------- | --------- | ------------------------- |
|
||||
| 0–80 km | Dark fiber / ZR | 100G–800G | <0.5 ms | Medium | AI clusters, storage sync |
|
||||
| 80–600 km | DWDM w/amp | 100G–400G | 0.5–3 ms | High | Regional DC |
|
||||
| 600–2000 km | Long-haul DWDM | 10G–400G | 3–15 ms | High | National backbone |
|
||||
| 2000–15000 km | Subsea cables | 10G–400G | 70–200 ms | Very high | Intercontinental |
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user