search

pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.16k stars 845 forks source link

How to 'Create model archive pod and run model archive file generation script' in the ‘User Guide’ #3004

Open Enochlove opened 6 months ago

Enochlove commented 6 months ago

🐛 Describe the bug

I'm reading the User Guide of KServe doc. One part of the 'Deploy a PyTorch Model with TorchServe InferenceService' is hard to understand.

3 'Create model archive pod and run model archive file generation script' 3.1 Create model archive pod and run model archive file generation script kubectl apply -f model-archiver.yaml -n kserve-test (https://kserve.github.io/website/0.11/modelserving/v1beta1/torchserve/model-archiver/)

Idk how to write the model-archiver.yaml and the model archive file generation script. I would be very grateful if anyone can help me！

Error logs

Not yet

Installation instructions

Yes yes

Model Packaing

Not yet

config.properties

No response

Versions

aiohttp==3.8.6 aiohttp-cors==0.7.0 aiorwlock==1.3.0 aiosignal==1.3.1 anyio==4.0.0 async-timeout==4.0.3 attrs==23.1.0 azure-core==1.29.5 azure-identity==1.15.0 azure-storage-blob==12.18.3 azure-storage-file-share==12.14.2 blessed==1.20.0

boto==31.28.73

botocore==1.31.73 cachetools==5.3.2 captum==0.6.0 certifi==2023.7.22 cffi==1.16.0 charset-normalizer==3.3.0 click==8.1.7 cloudevents==1.10.1 colorful==0.5.5 contourpy==1.1.1 cryptography==41.0.5 cuda-python==12.3.0 cycler==0.12.1 Cython==0.29.34 deprecation==2.1.0 distlib==0.3.7 enum-compat==0.0.3 exceptiongroup==1.1.3 fastapi==0.95.2 filelock==3.12.4 fonttools==4.43.1 frozenlist==1.4.0 fsspec==2023.9.2 google-api-core==2.12.0 google-auth==2.23.3 google-cloud-core==2.3.3 google-cloud-storage==1.44.0

google-crc==32c1.5.0

google-resumable-media==2.6.0 googleapis-common-protos==1.61.0 gpustat==1.1.1 grpcio==1.51.3 grpcio-tools==1.48.2

h==110.14.0

httpcore==0.16.3 httptools==0.6.1 httpx==0.23.3 huggingface-hub==0.17.3 idna==3.4 importlib-resources==6.1.0 isodate==0.6.1

Jinja==23.1.2

jmespath==1.0.1 jsonschema==4.19.2 jsonschema-specifications==2023.7.1 kiwisolver==1.4.5 kserve==0.11.1 kubernetes==28.1.0 MarkupSafe==2.1.3 matplotlib==3.8.0 mpmath==1.3.0 msal==1.24.1 msal-extensions==1.0.0 msgpack==1.0.7 multidict==6.0.4 networkx==3.1 numpy==1.24.3 nvidia-ml-py==12.535.108 oauthlib==3.2.2 opencensus==0.11.3 opencensus-context==0.1.3 orjson==3.9.10 packaging==23.2 pandas==2.1.2 Pillow==10.0.1

pip==23.3.1

platformdirs==3.11.0 portalocker==2.8.2 prometheus-client==0.13.1 protobuf==3.20.3 psutil==5.9.5 py-spy==0.3.14

pyasn==10.5.0

pyasn==1-modules0.3.0

pycparser==2.21 pydantic==1.10.13 PyJWT==2.8.0 pynvml==11.4.1 pyparsing==3.1.1 python-dateutil==2.8.2 python-dotenv==1.0.0 python-rapidjson==1.13 pytz==2023.3.post1 PyYAML==6.0 ray==2.4.0 referencing==0.30.2 regex==2023.10.3 requests==2.31.0 requests-oauthlib==1.3.1

rfc==39861.5.0

rpds-py==0.10.6 rsa==4.9

s==3transfer0.7.0

safetensors==0.4.0 setuptools==68.2.2 six==1.16.0 smart-open==6.4.0 sniffio==1.3.0 starlette==0.27.0 sympy==1.12 tabulate==0.9.0 timing-asgi==0.3.1 tokenizers==0.14.1 torch==2.1.0 torch-model-archiver==0.9.0 torch-workflow-archiver==0.2.11 torchaudio==2.1.0 torchdata==0.7.0 torchserve==0.9.0 torchtext==0.16.0 torchvision==0.16.0 tqdm==4.66.1 transformers==4.34.1 tritonclient==2.39.0 typing_extensions==4.8.0 tzdata==2023.3

urllib==31.26.18

uvicorn==0.19.0

uvloop==0.19.0

virtualenv==20.21.0 watchfiles==0.21.0 wcwidth==0.2.8 websocket-client==1.6.4 websockets==12.0 wheel==0.40.0 yarl==1.9.2 zipp==3.17.0

Repro instructions

None

Possible Solution

No response

agunapal commented 6 months ago

Hi @Enochlove You can follow this example https://github.com/pytorch/serve/blob/master/kubernetes/kserve/examples/mnist/MNIST.md

Enochlove commented 6 months ago

Hi @Enochlove You can follow this example https://github.com/pytorch/serve/blob/master/kubernetes/kserve/examples/mnist/MNIST.md

Hi @agunapal! Thanks for your reply. However, I still wanna try the example in the User Guide, cuz I'm trying to deploy a PyTorch Model with TorchServe InferenceService and Generate model archiver files for torchserve in Kubeflow . Can u help me in this kind of example? https://kserve.github.io/website/0.11/modelserving/v1beta1/torchserve/model-archiver/#33-delete-model-archiver

Enochlove commented 6 months ago

Hi @andyi2it Can u help me about this question? I would be very grateful.