search

hltcoe / sandle

Run a large language modeling SANDbox in your Local Environment
Other
7 stars 1 forks source link

Torch Serve #83

Closed ccmaymay closed 1 year ago

ccmaymay commented 1 year ago

https://pytorch.org/serve/ https://github.com/pytorch/serve/blob/master/examples/image_classifier/mnist/Docker.md https://github.com/pytorch/serve/tree/master/examples/Huggingface_Transformers https://github.com/pytorch/serve/issues/1579

ccmaymay commented 1 year ago

Difficult to tell how to merge this with fairscale, which manages its own parallelization using torch.distributed.

ccmaymay commented 1 year ago

Does not have clear support for model parallelism of large models, and other software exists that appears better (and more explicitly) suited to the task.