| Card | Arch | VRAM | Mem BW | NVFP4 | FP8 | INT8 | INT4 | BF16 | Software | Quirks & notes |
|---|---|---|---|---|---|---|---|---|---|---|
| RTX PRO 6000 | Blackwell | ~96 GB | ~1.8 TB/s | ✓ | ✓ | ✓ | ✓ | ✓ | CUDA 12.8+ · LightX2V / vLLM | Only tier that runs NVFP4. nvidia-smi --gpu-reset = Not Supported; WPR2 wedge clears only on HOST power-cycle. — vastnode002 |
| RTX PRO 5000 | Blackwell | 48 GB | ~1.3 TB/s (TBD) | ✓ | ✓ | ✓ | ✓ | ✓ | CUDA · vLLM 0.22.1 · on Vast | Vast clones ONE defjob to BOTH cards — no per-card unique jobs (caused the outage). — vastnode003 |
| RTX A5000 | Ampere | 24 GB | 768 GB/s | ✗ | ✗ | ✓ | ✓ | ✓ | driver 580.159.03 · CUDA 13.0 · diffusers / vLLM | No FP8 → use AWQ/bnb. Most mature stack: every kernel incl custom-CUDA TTS (NeMo). 24 GB tight → Omni capped at image:2. — r730a |
| RTX 4090 | Ada | 24 GB | ~1008 GB/s | ✗ | ✓ | ✓ | ✓ | ✓ | CUDA · vLLM / transformers / diffusers | Fleet workhorse. Native FP8 (Ada), no NVFP4 (Blackwell-only). Highest bandwidth of the 24 GB tier. 24 GB caps model size; runs on Vast as per-GPU defjobs (YuE etc.). — vastnode001 / infertha ×2 / am4-4u-newintown |
| RTX 4070S Ti | Ada | 16 GB | ~672 GB/s | ✗ | ✓ | ✓ | ✓ | ✓ | CUDA · transformers | Native FP8 (Ada); no NVFP4 (Blackwell-only). 16 GB caps model size (≈≤13B). — 3930a / 3930b |
| RX 7900 XTX | RDNA3 | 24 GB | 960 GB/s | ✗ | ✗ | ✓ | ✓ | ✓ | ROCm · llama.cpp-HIP · IPEX | Highest bandwidth in fleet. No FP8/FP4. bnb spotty; GGUF best via ROCm/HIP, Vulkan fallback. — vm905 |
| Arc Pro B70 | Battlemage Xe2 | 32 GB | 608 GB/s | ✗ | ✗ | ✓ | ✓ | ✓ | XPU · vLLM-XPU / IPEX-LLM / llm-scaler | Cheapest 32 GB ($949). int8 367 TOPS / int4 WOQ. GGUF → use Vulkan (SYCL near-CPU-slow). MoE/prefill slow; W8A8 unverified. NOT OWNED — evaluating |
✓ hardware-native △ emulated/partial ✗ not in silicon.
NVFP4 is Blackwell-only; FP8 needs Ada/Blackwell; Ampere/RDNA3/Battlemage top out at INT8. What's actually deployed per card is on Services.