dusty-nv
commented
10 months ago Sorry @ByerRA, text-generation-webui and the many libraries it requires don't support Jetson Nano and those containers are only available for JetPack 5. I do however have a transformers container for JetPack 4. Maybe one of the small Phi or Opt models could work in 4GB, but I'm not sure? I believe at one time I ran DistilBERT on Jetson Nano.
I executed the following build command on my Jetson Nano...
./build.sh text-generation-webuiAnd I get this error message immediately...
-- Package gptq-for-llama has missing dependencies, disabling... (couldn't find package: bitsandbytes) -- Package nanodb has missing dependencies, disabling... (couldn't find package: cuda-python) -- Package faiss:lite has missing dependencies, disabling... (couldn't find package: cuda-python) Traceback (most recent call last): File "/home/rbyer/miniforge3/envs/jetson-containers/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/rbyer/miniforge3/envs/jetson-containers/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/rbyer/jetson-containers/jetson_containers/build.py", line 96, in
build_container(args.name, args.packages, args.base, args.build_flags, args.simulate, args.skip_tests, args.test_only, args.push, args.no_github_api)
File "/home/rbyer/jetson-containers/jetson_containers/container.py", line 64, in build_container
packages = resolve_dependencies(packages)
File "/home/rbyer/jetson-containers/jetson_containers/packages.py", line 276, in resolve_dependencies
for dependency in find_package(package).get('depends', []):
File "/home/rbyer/jetson-containers/jetson_containers/packages.py", line 164, in find_package
raise ValueError(f"couldn't find package: {package}")
ValueError: couldn't find package: text-generation-webui