Open georgejeno8 opened 8 months ago
Looking at https://github.com/huggingface/text-generation-inference/tree/v1.1.1/server and https://github.com/huggingface/text-generation-inference/tree/v1.2.0/server it seems that they moved from a single requirements.txt file to requirements_common.txt and requirements_cuda.txt, does that need to be reflected in the build script?
@georgejeno8 probably, but I don't really maintain text-generation-inference package as its heavyweight and I don't use it. Happy to accept PR's though!
Most likely you need pytorch:2.0-distributed
rather than pytorch:2.0
Hello,
I followed the system setup instructions and tried to build the text-generation-inference container on my Jetson Orin 8GB running JetPack 5.1, but I seem to be running into the following error:
When I try to run from one of the prebuilt images instead everything starts successfully, but the following error occurs when trying to run the inference server with
text-generation-launcher
:Is there something incorrect with how my Jetson Orin is configured? Thank you very much for the help!