Bug Fix: Check for explicit cli device (fast)

pytorch / torchchat

Run PyTorch LLMs locally on servers, desktop and mobile

BSD 3-Clause "New" or "Revised" License

3.4k stars 224 forks source link

Bug Fix: Check for explicit cli device (fast) #1374

Closed Jack-Khuu closed 1 week ago

Jack-Khuu commented 1 week ago

This fixes a bug where a user provided "fast" device arg falls through without being converted to a device. This correctly catches this case.

Bug introduced in https://github.com/pytorch/torchchat/commit/93f713f12507b5cef18a42c411030c90b9326369,

Command

python3 torchchat.py generate llama3.1 --prompt "write me a story about a boy and his bear" --device fast

Original Failure:

RuntimeError: don't know how to restore data location of torch.storage.UntypedStorage (tagged with fast)

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1374

:page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

GLIBC not found in Nova workflows

:white_check_mark: No Failures

As of commit 703786c89b4c5e2abcf296edeb23537901efb76d with merge base 2cf1a17a74799cc797a525eafa27da575d568193 (): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.