search

mlc-ai / relax

Apache License 2.0
149 stars 75 forks source link

Cache request improvements #311

Closed DavidGOrtega closed 4 months ago

DavidGOrtega commented 8 months ago

IMPORTANT WIP 🙏 Model weights at full speed 🚀 I think this is a great differentiator over monolithic gguf

https://github.com/mlc-ai/relax/assets/414967/9e6998f3-7ef8-4c4a-b15e-c1a4c420fff1

@CharlieFRuan @tqchen

Loader Logger must be changed to report once the file is downloaded and not in advance.

CharlieFRuan commented 8 months ago

This is great, really appreciate the changes! Could you make a PR to TVM's main branch instead? We can review there!

DavidGOrtega commented 8 months ago

This is great, really appreciate the changes! Could you make a PR to TVM's main branch instead? We can review there!

Of course! I need to clone https://github.com/apache/tvm right? At what point is it going to be merged and deprecate this one?

CharlieFRuan commented 8 months ago

Yep, it's that repo! It will be picked to this relax repo soon once it's merged there.