triton-inference-server / tensorrtllm_backend

The Triton TensorRT-LLM Backend
Apache License 2.0
711 stars 108 forks source link

fill_template.py and gpu_device_ids #616

Open Alireza3242 opened 1 month ago

Alireza3242 commented 1 month ago

System Info

3090

Who can help?

@juney-nvidia @byshiue @schetlur-nv

Information

Tasks

Reproduction

I want to use both gpu_device_ids 0 and 1:

python3 fill_template.py --in_place config.pbtxt gpu_device_ids:0,1

Expected behavior

work

actual behavior

Traceback (most recent call last):
  File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
    main(**vars(args))
  File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
    key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
  File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
    main(**vars(args))
  File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
    key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
  File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
    main(**vars(args))
  File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
    key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
  File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
    main(**vars(args))
  File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
    key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Traceback (most recent call last):
  File "/app/src/config_modifier/../tools/fill_template.py", line 38, in <module>
    main(**vars(args))
  File "/app/src/config_modifier/../tools/fill_template.py", line 12, in main
    key, value = sub.split(":")
ValueError: not enough values to unpack (expected 2, got 1)

additional notes

the problem is "," in "0,1"