Open Delaunay opened 3 weeks ago
================= Benchmark results ================= bench | fail | n | perf | sem% | std% | peak_memory | score | weight brax | 1 | 1 | 4822.59 | 0.6% | 1.4% | nan | 0.00 | 1.00 diffusion-gpus | 0 | 1 | 360.89 | 0.1% | 1.0% | 60298 | 360.89 | 1.00 diffusion-single | 0 | 1 | 362.63 | 0.1% | 1.0% | 60336 | 362.63 | 0.00 dinov2-giant-gpus | 0 | 1 | 864.20 | 0.4% | 2.9% | 72102 | 864.20 | 1.00 dinov2-giant-single | 0 | 8 | 102.86 | 0.4% | 9.5% | 74544 | 831.34 | 0.00 bf16 | 0 | 8 | 784.40 | 0.2% | 5.8% | 1548 | 6308.94 | 0.00 fp16 | 0 | 8 | 799.06 | 0.2% | 6.3% | 1548 | 6431.99 | 0.00 fp32 | 0 | 8 | 51.94 | 0.0% | 0.7% | 1932 | 415.81 | 0.00 tf32 | 0 | 8 | 406.56 | 0.2% | 5.5% | 1932 | 3258.38 | 0.00 dimenet | 0 | 8 | 483.93 | 0.7% | 15.0% | 7456 | 3905.80 | 0.00 bert-fp16 | 0 | 8 | 457.20 | 0.9% | 14.2% | nan | 3744.58 | 0.00 bert-fp32 | 0 | 8 | 111.46 | 0.4% | 6.5% | 6436 | 901.45 | 0.00 bert-tf32 | 0 | 8 | 243.67 | 0.7% | 10.3% | nan | 1983.29 | 0.00 bert-tf32-fp16 | 0 | 8 | 457.23 | 0.9% | 14.3% | nan | 3744.93 | 3.00 reformer | 0 | 8 | 103.20 | 0.4% | 8.2% | 25208 | 833.78 | 1.00 t5 | 0 | 8 | 89.13 | 0.4% | 9.7% | 9208 | 721.16 | 2.00 whisper | 0 | 8 | 878.49 | 0.5% | 10.7% | 1026 | 7113.36 | 1.00 lightning | 0 | 8 | 1222.54 | 0.6% | 13.7% | 27078 | 9900.16 | 0.00 lightning-gpus | 0 | 1 | 9849.93 | 0.7% | 5.7% | 30516 | 9849.93 | 1.00 llama | 0 | 8 | 768.96 | 4.8% | 85.9% | 27608 | 5787.01 | 1.00 llm-full-mp-gpus | 0 | 1 | 513.47 | 2.4% | 12.9% | 49382 | 513.47 | 1.00 llm-lora-ddp-gpus | 0 | 1 | 29652.09 | 0.4% | 2.2% | 39352 | 29652.09 | 1.00 llm-lora-mp-gpus | 0 | 1 | 3810.83 | 1.8% | 9.3% | 55454 | 3810.83 | 1.00 llm-lora-single | 0 | 8 | 5048.74 | 0.3% | 4.2% | 49732 | 40488.86 | 1.00 recursiongfn | 0 | 8 | 12029.94 | 0.8% | 17.8% | 10692 | 97098.93 | 0.00 super-slomo | 0 | 8 | 87.64 | 0.7% | 14.5% | 67926 | 706.49 | 1.00 focalnet | 0 | 8 | 649.43 | 0.6% | 13.0% | 24524 | 5263.90 | 2.00 torchatari | 0 | 8 | 8370.17 | 0.3% | 7.1% | 3690 | 66911.20 | 0.00 convnext_large-fp16 | 0 | 8 | 663.05 | 1.0% | 15.2% | nan | 5440.84 | 0.00 convnext_large-fp32 | 0 | 8 | 129.51 | 0.6% | 9.4% | 47170 | 1052.58 | 0.00 convnext_large-tf32 | 0 | 8 | 239.24 | 0.8% | 12.6% | 49506 | 1955.14 | 0.00 convnext_large-tf32-fp16 | 0 | 8 | 664.53 | 1.0% | 15.2% | nan | 5452.64 | 3.00 regnet_y_128gf | 0 | 8 | 187.71 | 0.4% | 9.0% | 28740 | 1517.87 | 2.00 resnet152-ddp-gpus | 0 | 1 | 8172.97 | 0.1% | 0.4% | 30964 | 8172.97 | 0.00 resnet50 | 0 | 8 | 1804.63 | 0.6% | 13.0% | 13340 | 14621.76 | 1.00 resnet50-noio | 0 | 8 | 2029.43 | 0.4% | 7.8% | 27354 | 16386.23 | 0.00
Delaunay
commented
3 weeks ago Delaunay
commented
3 weeks ago