[Tools] Add convert tool for Llama models quantized by AutoGPTQ

[Duyi-Wang]

It's preferable to an additional param for quantification rather than creating a new converter, as this approach allows us to integrate new quantification methods in the future, like AWQ

[Duyi-Wang]

And update the description of Converter in README.

[Duyi-Wang]

How about quantization:Optional[str]= "gpqt"? use_gptq is inconvenient to add new types in future

[xiangzez]

For documentation, I think we need a separate tutorial page for quantization. @miaojinc already wrote a doc and we should update this doc and put it in main branch.

[miaojinc]

For documentation, I think we need a separate tutorial page for quantization. @miaojinc already wrote a doc and we should update this doc and put it in main branch.

Sure, I will update the doc after this PR merged.

[Duyi-Wang]

For documentation, I think we need a separate tutorial page for quantization. @miaojinc already wrote a doc and we should update this doc and put it in main branch.

Sure, I will update the doc after this PR merged.

@miaojinc Could you add it to our docs, source code is under docs branch.

[miaojinc]

For documentation, I think we need a separate tutorial page for quantization. @miaojinc already wrote a doc and we should update this doc and put it in main branch.

Sure, I will update the doc after this PR merged.

@miaojinc Could you add it to our docs, source code is under docs branch.

Yes, sure. I will do that in a new pull request for quantization document.

[changqi1]

@xiangzez CI failed on baichuan model?

[xiangzez]

@changqi1 CI issue should be fixed in #287

[Duyi-Wang]

@changqi1 CI issue should be fixed in #287

rebase or merge main branch？

[changqi1]

We could rebase to check this PR's status.