search

JinYuanLi0012 / PGIM

[EMNLP 2023 Findings] Prompting Chatgpt in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge
19 stars 0 forks source link

requests.exceptions.HTTPError: SequenceLabelingPreprocessor: 404 Client Error: Not Found for url: http://www.modelscope.cn/api/v1/models/xlm-roberta-large/revisions?EndTime=1688313600 #9

Closed hhy150 closed 2 months ago

hhy150 commented 2 months ago

模型地址现在貌似找不到了?不知道有没有其他下载地址,下载到本地,然后到哪里改下代码呢?

JinYuanLi0012 commented 2 months ago

我们团队成员两天前在新机器上顺利复现了该模型，所以应该不是模型地址的原因，可以尝试排查一下网络？

hhy150 commented 2 months ago

image 你好哇，我直接搜索也是报这个错呢，我看前面获取数据貌似也是可以的。 然后我的整个报错信息是这样的：

2024-06-13 22:32:41,955 - modelscope - INFO - PyTorch version 1.13.1 Found. 2024-06-13 22:32:41,955 - modelscope - INFO - Loading ast index from /data2/hhy/.cache/modelscope/ast_indexer 2024-06-13 22:32:41,955 - modelscope - INFO - No valid ast index found from /data2/hhy/.cache/modelscope/ast_indexer, generating ast index from prebuilt! 2024-06-13 22:32:42,040 - modelscope - INFO - Loading done! Current index file version is 1.7.1, with md5 22229ed48df4c136e5165c03566fd2b7 and a total number of 861 components indexed 2024-06-13 22:32:44,536 - modelscope - WARNING - The reference has been Deprecated in modelscope v1.4.0+, please use from modelscope.msdatasets.dataset_cls.custom_datasets import TorchCustomDataset 2024-06-13 22:32:44,747 - INFO - adaseq.data.dataset_manager - Will use a custom loading script: /data2/hhy/MNER/adaseq/adaseq/data/dataset_builders/named_entity_recognition_dataset_builder.py Downloading data: 525kB [00:00, 7.11MB/s] Generating train split: 4000 examples [00:00, 7265.71 examples/s] Generating valid split: 1000 examples [00:00, 13259.06 examples/s] Generating test split: 3257 examples [00:00, 13454.56 examples/s] 2024-06-13 22:32:47,306 - INFO - adaseq.data.datasetmanager - First sample in train set: {'id': '0', 'tokens': ['RT', '@JayKenMinaj', '', ':', 'Me', 'outside', 'of', 'where', 'George', 'Zimmerman', 'got', 'shot', 'at', '.', 'You', 'know', 'God', 'is', 'so', 'good', '.', 'http://t.co/Z3neVBQ7vF'], 'spans': [{'start': 8, 'end': 10, 'type': 'PER'}], 'mask': [True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True]} Counting labels by count_span_labels: 100%|████████████████████████████████████| 4000/4000 [00:00<00:00, 10238.49 examples/s] Counting labels by count_span_labels: 100%|████████████████████████████████████| 1000/1000 [00:00<00:00, 10096.76 examples/s] 2024-06-13 22:32:47,798 - INFO - adaseq.data.preprocessors.sequence_labeling_preprocessor - label_to_id: {'O': 0, 'B-LOC': 1, 'I-LOC': 2, 'E-LOC': 3, 'S-LOC': 4, 'B-ORG': 5, 'I-ORG': 6, 'E-ORG': 7, 'S-ORG': 8, 'B-OTHER': 9, 'I-OTHER': 10, 'E-OTHER': 11, 'S-OTHER': 12, 'B-PER': 13, 'I-PER': 14, 'E-PER': 15, 'S-PER': 16} 2024-06-13 22:33:27,979 - ERROR - modelscope - Authentication token does not exist, failed to access model xlm-roberta-large which may not exist or may be private. Please login first. 2024-06-13 22:33:27,979 - ERROR - modelscope - Response details: {'Code': 10010205001, 'Message': '获取模型信息失败，信息：record not found', 'RequestId': '5cf2843a-ad38-4792-b054-4815881ce4d8', 'Success': False}

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) socket.timeout: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connectionpool.py", line 793, in urlopen response = self._make_request( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connectionpool.py", line 467, in _make_request self._validate_conn(conn) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn conn.connect() File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connection.py", line 616, in connect self.sock = sock = self._new_conn() File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connection.py", line 207, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f5b1a8a8af0>, 'Connection to huggingface.co timed out. (connect timeout=10)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/requests/adapters.py", line 589, in send resp = conn.urlopen( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/connectionpool.py", line 847, in urlopen retries = retries.increment( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /xlm-roberta-large/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f5b1a8a8af0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 395, in _request_wrapper response = get_session().request(method=method, url=url, params) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/utils/_http.py", line 66, in send return super().send(request, args, **kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/requests/adapters.py", line 610, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /xlm-roberta-large/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f5b1a8a8af0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 95b7d315-4d56-4dcc-9d8f-a9968d366891)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/utils/hub.py", line 417, in cached_file resolved_file = hf_hub_download( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir _raise_on_head_call_error(head_call_error, force_download, local_files_only) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 1826, in _raise_on_head_call_error raise LocalEntryNotFoundError( huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data2/hhy/MNER/adaseq/adaseq/data/tokenizer.py", line 19, in build_tokenizer return AutoTokenizer.from_pretrained(model_name_or_path, use_fast=use_fast, kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 658, in from_pretrained config = AutoConfig.from_pretrained( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 944, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/configuration_utils.py", line 574, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/configuration_utils.py", line 629, in _get_config_dict resolved_config_file = cached_file( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/transformers/utils/hub.py", line 452, in cached_file raise EnvironmentError( OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like xlm-roberta-large is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/utils/registry.py", line 212, in build_from_cfg return obj_cls(args) File "/data2/hhy/MNER/adaseq/adaseq/data/preprocessors/sequence_labeling_preprocessor.py", line 38, in init super().init(model_dir, label_to_id=label_to_id, return_offsets=True, kwargs) File "/data2/hhy/MNER/adaseq/adaseq/data/preprocessors/nlp_preprocessor.py", line 71, in init self.tokenizer = build_tokenizer( File "/data2/hhy/MNER/adaseq/adaseq/data/tokenizer.py", line 22, in build_tokenizer model_name_or_path = get_or_download_model_dir(model_name_or_path) File "/data2/hhy/MNER/adaseq/adaseq/utils/hub_utils.py", line 14, in get_or_download_model_dir model_cache_dir = snapshot_download( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/hub/snapshot_download.py", line 96, in snapshot_download revision = _api.get_valid_revision( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/hub/api.py", line 464, in get_valid_revision revisions = self.list_model_revisions( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/hub/api.py", line 433, in list_model_revisions handle_http_response(r, logger, cookies, model_id) File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/hub/errors.py", line 90, in handle_http_response raise error File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/hub/errors.py", line 82, in handle_http_response response.raise_for_status() File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://www.modelscope.cn/api/v1/models/xlm-roberta-large/revisions?EndTime=1688313600

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/data2/hhy/MNER/adaseq/scripts/train.py", line 38, in train_model_from_args(args) File "/data2/hhy/MNER/adaseq/adaseq/commands/train.py", line 85, in train_model_from_args train_model( File "/data2/hhy/MNER/adaseq/adaseq/commands/train.py", line 159, in train_model trainer = build_trainer_from_partial_objects( File "/data2/hhy/MNER/adaseq/adaseq/commands/train.py", line 199, in build_trainer_from_partial_objects preprocessor = build_preprocessor(config.preprocessor, labels=dm.labels) File "/data2/hhy/MNER/adaseq/adaseq/data/preprocessors/nlp_preprocessor.py", line 251, in build_preprocessor return ms_build_preprocessor(config, field_name, kwargs) # type: ignore File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/preprocessors/builder.py", line 21, in build_preprocessor return build_from_cfg( File "/data2/hhy/anaconda3/envs/mner/lib/python3.8/site-packages/modelscope/utils/registry.py", line 215, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') requests.exceptions.HTTPError: SequenceLabelingPreprocessor: 404 Client Error: Not Found for url: http://www.modelscope.cn/api/v1/models/xlm-roberta-large/revisions?EndTime=1688313600

JinYuanLi0012 commented 2 months ago

是否可能是因为连不上huggingface所导致的？

hhy150 commented 2 months ago

刚刚查到一个解决方案：https://developer.aliyun.com/ask/626669，解决了这个问题，2024-06-13 22:33:27,979 - ERROR - modelscope - Authentication token does not exist, failed to access model xlm-roberta-large which may not exist or may be private. Please login first. 之后确实出现了huggingface的问题，然后看你之前说的加上了环境变量就可以了。感谢感谢！！ export HF_ENDPOINT=https://hf-mirror.com/