search

tunib-ai / parallelformers

Parallelformers: An Efficient Model Parallelization Toolkit for Deployment
https://tunib-ai.github.io/parallelformers
Apache License 2.0
776 stars 61 forks source link

Recommended way for cleaning up? #24

Open creatorrr opened 2 years ago

creatorrr commented 2 years ago

Hi there!

Thanks for the awesome work on this lib! Just wanted to ask what the recommended way is to clean up a loaded model that has been parallelized using this library. What method should be called to clean up all the resources, move data out of the GPU, empty cuda cache and shut down the master process?

Tried to run this but it hanged:

model = ...
p = parallelize(
    model, 
    num_gpus=2, 
    fp16=True, 
    verbose="simple",
)

# Do some inference

p.deparallelize()  # --> This hanged