search

aws-neuron / transformers-neuronx

Apache License 2.0
88 stars 25 forks source link

User feedback when compiling and reloading a large model #76

Open dacorvo opened 5 months ago

dacorvo commented 5 months ago

Converting a model to neuron takes a lot of time, and even though the compilation logs indicate that something is going on, the user doesn't have any clue of how long it will take.

It would be great to be able to get some hints on the progression, for instance by indicating how much graphs have been compiled so far out of the total number of graphs to compile.

Reloading a neuron model from compiled artifacts can take several minutes for large models, but there are no compilation logs this time.

It would be useful to be able to provide user feedback during the process.

awsilya commented 5 months ago

@dacorvo thank you for the feedback. Yes we will work on improving the experience.