Can I save the compressed model for direct inference only?

[SparkJiao]

Excellent work

May I know if I could save the compressed model locally for further inference, e.g., combined with lora adapters?

I see the NotImplementedError raised so I'm not sure if there is something should be noted.

Thanks!

[Vahe1994]

Hey!

Thanks for the interest in our work!

We've released only the evaluation code for now (you can use it to evaluate the compressed model's quality).

We'll add code for efficient inference soon (in ~2 weeks).

[SparkJiao]

Thanks for your clarification!

[singingtower]

Hey!

Thanks for the interest in our work!

We've released only the evaluation code for now (you can use it to evaluate the compressed model's quality).

We'll add code for efficient inference soon (in ~2 weeks).

has the inferance code released by now? failed to find it!