Initialize a model that takes no host or CPU memory (for cases where the model is larger than the host memory)
Replay the operations that were played while initializing a model or a partition of the model
APIs
from pipegoose.utils import lazy_init
# load the model from `transformers`
with lazy_init(parallel_context):
model = TensorParallel(model, parallel_context).parallelize()
model = PipelineParallel(model, parallel_context).parallelize()
model = DataParallel(model, parallel_context).parallelize()
logits = model(inputs)
APIs
Reading