Bug fix: Enable fast to override quantize json

[Jack-Khuu]

Follow up fix for: https://github.com/pytorch/torchchat/commit/46977645de6e9e29e58fada7d600c1930ed6f67b

We want to allow "fast" to work as a manual overwrite of a --quantize accelerator config, which the previous PR disabled while fixing a different bug

---

No overwrite

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"mps"}}'

NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

Overwrite with device arg

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"mps"}}' --device cpu

overriding json-specified device mps with cli device cpu
NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=cpu Apple M1 Max

Overwrite with fast arg

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --device fast

overriding json-specified device cpu with cli device mps
NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

Run with fast arg (same as running without device arg)

python3 torchchat.py generate llama3.1 --device fast

NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1377

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [DomainsOnly] Jobs fail with GLIBC version not found

:white_check_mark: No Failures

As of commit 8db4b728279a1acbbbd0e10d0dc8b0f7f22ed411 with merge base 46977645de6e9e29e58fada7d600c1930ed6f67b (): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

cc: @mikekgfb