bug: load local model

[aarnphm]

Describe the bug

Somewhere along the way with the tag refactoring, --model-id failed to load model from local path.

We have an internal tracking issue on reworking the tag generation, which will address this problem.

cc @larme

To reproduce

No response

Logs

No response

Environment

Persistent throughout different python version.

System information (Optional)

No response

[p208p2002]

As a temporary solution, setting the environment variable TRANSFORMERS_OFFLINE=1 and use absolute path that works for me.

[yingjie-han]

As a temporary solution, setting the environment variable TRANSFORMERS_OFFLINE=1 and use absolute path that works for me.

It is not working for me. $ HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 openllm start baichuan --model-id /home/yingjie/openllm/baichuan2-13b --backend pt --debug Traceback (most recent call last): File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/transformers/init.py", line 147, in get model = bentoml.models.get(llm.tag) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/simpledi/init.py", line 139, in return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs)) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/bentoml/models.py", line 45, in get return _model_store.get(tag) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/bentoml/_internal/store.py", line 158, in get raise NotFound( bentoml.exceptions.NotFound: Model 'pt-baichuan2-13b:08c4d4d5d8625c6702b44beca2570febec83a4ae' is not found in BentoML store <osfs '/root/bentoml/models'>, you may need to run bentoml models pull first

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 416, in import_command _ref = openllm.serialisation.get(llm) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/init.py", line 75, in caller return getattr(importlib.import_module(f'.{serde}', name), fn)(llm, *args, **kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/transformers/init.py", line 155, in get raise openllm.exceptions.OpenLLMException(f'Failed while getting stored artefact (lookup for traceback):\n{err}') from err openllm_core.exceptions.OpenLLMException: Failed while getting stored artefact (lookup for traceback): Model 'pt-baichuan2-13b:08c4d4d5d8625c6702b44beca2570febec83a4ae' is not found in BentoML store <osfs '/root/bentoml/models'>, you may need to run bentoml models pull first

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/transformers/init.py", line 147, in get model = bentoml.models.get(llm.tag) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/simpledi/init.py", line 139, in return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs)) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/bentoml/models.py", line 45, in get return _model_store.get(tag) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/bentoml/_internal/store.py", line 158, in get raise NotFound( bentoml.exceptions.NotFound: Model 'pt-baichuan2-13b:08c4d4d5d8625c6702b44beca2570febec83a4ae' is not found in BentoML store <osfs '/root/bentoml/models'>, you may need to run bentoml models pull first

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yingjie/openllm/v_openllm_bc/bin/openllm", line 8, in sys.exit(cli()) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 196, in wrapper return_value = func(*args, *attrs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 178, in wrapper return f(*args, attrs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/_factory.py", line 179, in start_cmd llm = openllm.utils.infer_auto_class(env['backend_value']).for_model(model, File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/models/auto/factory.py", line 52, in for_model if ensure_available: llm.save_pretrained() File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/_llm.py", line 672, in save_pretrained return openllm.import_model(self.config['start_name'], File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/_sdk.py", line 262, in _import_model return import_command.main(args=args, standalone_mode=False) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 196, in wrapper return_value = func(*args, attrs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, *kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 178, in wrapper return f(args, attrs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/cli/entrypoint.py", line 422, in import_command _ref = openllm.serialisation.get(llm, auto_import=True) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/init.py", line 75, in caller return getattr(importlib.import_module(f'.{serde}', name), fn)(llm, *args, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/transformers/init.py", line 154, in get if auto_import: return import_model(llm, trust_remote_code=llm.trust_remote_code) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/simpledi/init.py", line 139, in return func(*_inject_args(bind.args), _inject_kwargs(bind.kwargs)) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/openllm/serialisation/transformers/init.py", line 94, in import_model tokenizer = infer_tokenizers_from_llm(llm).from_pretrained(llm.model_id, trust_remote_code=trust_remote_code, hub_attrs, *tokenizer_attrs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, inputs, kwargs) File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2017, in from_pretrained return cls._from_pretrained( File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2249, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/root/.cache/huggingface/modules/transformers_modules/baichuan2-13b/tokenization_baichuan.py", line 71, in init super().init( File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/transformers/tokenization_utils.py", line 367, in init self._add_tokens( File "/home/yingjie/openllm/v_openllm_bc/lib64/python3.8/site-packages/transformers/tokenization_utils.py", line 467, in _add_tokens current_vocab = self.get_vocab().copy() File "/root/.cache/huggingface/modules/transformers_modules/baichuan2-13b/tokenization_baichuan.py", line 105, in get_vocab vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)} File "/root/.cache/huggingface/modules/transformers_modules/baichuan2-13b/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

[aarnphm]

I believe this has been addressed. The Baichuan issue will be tracked separately. cc @larme