Confirmed with a test deploy by using mpt_7b_instruct_ft.yaml with gcp_path
2023-07-24 13:56 ❯ mcli get deployment logs mpt-7b-instruct-ft-5a2e3q
Cloning into 'examples'...
Note: switching to 'd1cd9293b3eb943d38e912a3b9f4abd8bad52b5b'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at d1cd929 Add GCP downloader in example (#419)
/code
Cloning into 'llm-foundry'...
Note: switching to '496b50bd588b1a7231fe54b05d70babb3620fc72'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 496b50b Add trust remote code for tokenizer in inference conversion script (#446)
/code
/code/FasterTransformer/examples/pytorch/gpt/utils/gpt.py:221: SyntaxWarning: assertion is always true, perhaps remove parentheses?
assert(self.pre_embed_idx < self.post_embed_idx, "Pre decoder embedding index should be lower than post decoder embedding index.")
Downloading model from path: gs://mosaicml-mcloud-tests/sg_test
Downloading /tmp/mpt/local_model/checkpoints-1671246524-rare-albatross-ep0-ba12-rank0.pt...
Converting model to FT format
Traceback (most recent call last):
File "/code/inference_go/downloader.py", line 83, in <module>
download_func(**model_config.download_parameters)
File "/code/examples/examples/inference-deployments/mpt/mpt_ft_handler.py", line 122, in download_convert
convert_mpt_to_ft(model_name_or_path, LOCAL_CHECKPOINT_DIR, gpus,
File "/code/llm-foundry/scripts/inference/convert_hf_mpt_to_ft.py", line 200, in convert_mpt_to_ft
model = transformers.AutoModelForCausalLM.from_pretrained(
File "/usr/lib/python3/dist-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/usr/lib/python3/dist-packages/transformers/models/auto/configuration_auto.py", line 908, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/usr/lib/python3/dist-packages/transformers/configuration_utils.py", line 573, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/usr/lib/python3/dist-packages/transformers/configuration_utils.py", line 628, in _get_config_dict
resolved_config_file = cached_file(
File "/usr/lib/python3/dist-packages/transformers/utils/hub.py", line 380, in cached_file
raise EnvironmentError(
OSError: /tmp/mpt/local_model does not appear to have a file named config.json. Checkout 'https://huggingface.co//tmp/mpt/local_model/None' for available files.
Confirmed with a test deploy by using mpt_7b_instruct_ft.yaml with gcp_path