Methodology: AigenRig Bench Protocol v1.0
Every benchmark on AigenRig follows one fixed protocol: same workflows, same seed, same quantization, only the GPU changes. That makes cards comparable across reviews and makes every number reproducible. This page is a high-level summary; the full lab protocol (environment checklist, exact model files, screenshot and archive rules, CSV template) is maintained in the site repository at docs/AigenRig_GPU_Test_Protocol_v1.md.
Environment
- ComfyUI pinned to a recorded version / git hash (v1.0 baseline: 0.3.59), PyTorch with CUDA 12.x matched to the driver; the same software stack within any comparison batch.
- NVIDIA driver and CUDA version recorded with
nvidia-smibefore every test set. - Default launch flags only — no
--lowvram,--novram, or--cpuin the main table. - Platforms: local rigs and cloud GPUs (e.g. RunPod). The platform, instance type, and price are recorded per row.
Run rules
- Seed 42, batch size 1, no other GPU workloads running.
- 1 warm-up run (not timed), then 3 timed runs; we report the median.
- Timing is wall-clock from sampler start through VAE decode, read from ComfyUI’s “Prompt executed in … seconds”. Model load time is excluded.
Fixed workloads
| Workload | Model file | Settings |
|---|---|---|
| FLUX.1 [dev] FP8 (image, main) | flux1-dev-fp8.safetensors | 1024×1024 · 20 steps · euler / simple · guidance 3.5 |
| SDXL 1.0 base (image, reference) | sd_xl_base_1.0.safetensors | 1024×1024 · 20 steps · euler / normal · CFG 7.0 |
| Wan 2.1 T2V 480p (video, consumer cards) | wan2.1_t2v_1.3B_fp16 | 832×480 · 81 frames (~5 s @ 16 fps) · 30 steps · uni_pc / simple · CFG 6 · shift 8 |
| Wan 2.1 T2V 720p (video, 24GB+ cards) | wan2.1_t2v_14B_fp8_scaled | 1280×720 · 81 frames · 50 steps · uni_pc / simple · CFG 5 · shift 5 |
Image tests share one prompt; both video tests share one prompt and negative prompt.
What we measure
- Seconds per image or seconds per ~5 s clip (median of 3).
- Peak VRAM: maximum
memory.usedsampled during the timed run. - Power: average GPU
power.drawat steady state (not rated TDP). - Cost per 100 jobs:
- Local:
(seconds/3600) × (W/1000) × $/kWh × 100(default placeholder $0.15/kWh, stated on each page) - Cloud:
(seconds/3600) × $/hr × 100, using the actual instance price
- Local:
Pass / fail
- PASS: warm-up and all 3 timed runs complete at the specified resolution and frame count.
- PASS with offload: only completes with
--lowvram, node offload, or a GGUF fallback. Listed in a separate row, never mixed into the main ranking. - FAIL (OOM / timeout / quality): recorded as a fail for that protocol row; it does not enter the speed ranking.
Versioning
Changing steps, resolution, or model files bumps the protocol version (v1.1, …). Older articles keep their v1.0 footnote so historical data stays comparable.
Status
First test wave: RTX 3060 12GB → used RTX 3090 → RTX 4090 (RunPod). Until those runs are published, score cells on this site read TBD. See Best GPU for AI Image & Video (2026) and How Much VRAM for Flux, SDXL & Wan.
Spotted a problem with a number? See Contact.