For quality checks reference should be taken from actual PyTorch version of model

premAI-io / benchmarks

🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.

MIT License

117 stars 5 forks source link

For quality checks reference should be taken from actual PyTorch version of model #162

Closed Anindyadeep closed 2 months ago

Anindyadeep commented 2 months ago

In order to do quick emprical testing, the ground truth of the model should be the model output from the raw float32 weights, which can be done from HF Transformers.

Anindyadeep commented 2 months ago

Issue has been resolved through PR #167 and #116. So now we can close this issue.