LazyModel makes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reason LazyModel exists is so that we can spawn new processes and share a PipelineConfig instance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.
Idea
To get rid of LazyModel without sacrificing multiprocessing we need to lighten the content of PipelineConfig, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.
This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.
Problem
LazyModelmakes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reasonLazyModelexists is so that we can spawn new processes and share aPipelineConfiginstance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.Idea
To get rid of
LazyModelwithout sacrificing multiprocessing we need to lighten the content ofPipelineConfig, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.