-
hi
when you compute the FLOPS in table 6 for baseline models such as ViLBERT, do you also include the FLOPS computation of feature extraction models?
-
Sample Output for GPT 18.4B Parameter :
**Profiler reports FLOPS per GPU as 13.36 TFLOPS, whereas the log prints the FLOPS per GPU as 125.18 TFLOPs**
Profiler printed Samples/s is 49.55 and that p…
-
I want to get FLOPs for my model, using the profile of the thop library, but the number of model parameters obtained from the thop library is smaller than n_parameters, why is that?
-
https://huggingface.co/collections/allenai/olmo-2-674117b93ab84e98afc72edc
## Evaluation
Core model results for OLMo 2 7B and 13B models are found below.
| Model | Train FLOPs | Average | ARC…
-
Very excellent work, but we would like to further evaluate the efficacy of the model.
How do we test the FLOPs of the model?
-
When using linen's remat_scan functionality for constant-time compilation w.r.t. depth, our team is encountering an error in the estimated FLOP count for the compiled training step.
In particular,…
-
What methodology do you employ to evaluate the FLOPs of the models illustrated in the succeeding visual representation?
, but the current code doesn't seem to support MM…
-
Wrong flops count if the model is compiled with `torch.compile`:
1. Flops of modules in `torch.nn`, for example, `nn.Linear`, `nn.Conv2d` are tripled.
2. Flops of custom modules are not counted.
…
-
### 🚀 Motivation and context
Is it possible to correlate kernel distribution with ranges annotated either through `torch.cuda.nvtx` or `torch.profiler.profile`?
The use case is model architectur…