AutoGPTQ Mistral, Memory profiling support and empirical quality checks

premAI-io / benchmarks

🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.

MIT License

117 stars 5 forks source link

AutoGPTQ Mistral, Memory profiling support and empirical quality checks #163

Closed Anindyadeep closed 2 months ago

Anindyadeep commented 2 months ago

This PR introduces all the changes by PR #167 and integrates those in AutoGPTQ. AutoGPTQ README now has quality checks table for both Llama 2 and Mistral

nsosio commented 2 months ago

@Anindyadeep can you solve conflicts?

Anindyadeep commented 2 months ago

@Anindyadeep can you solve conflicts?

Yes, will ping you once that done

Anindyadeep commented 2 months ago

This is one is ready to merge

cc: @nsosio :)