search

kongds / Prompt-BERT

PromptBERT: Improving BERT Sentence Embeddings with Prompts
332 stars 33 forks source link

Invalid address about configs. #32

Closed Louis-Jackson closed 1 year ago

Louis-Jackson commented 1 year ago

Hello,

I was fortunate to read your paper, and the experienmental results are really exicting.

However, when I am running the code you provided in GitHub, there are some issues happened. It reminded me that the confit.json which should be downloaded in "https://huggingface.co/result/unsup-bert_s0/resolve/main/config.json" is not found.

How could I solve this problem?

Looking forward to your reply, thanks!

kongds commented 1 year ago

Thanks for your interesting to our paper.

We do not upload our model weigh on HuggingFace yet.But you provide download link via gdriver in readme. you can download it via https://drive.google.com/file/d/1n9FULUIRBhmhvaSQPaOnsudb_CVZyBli/view

Louis-Jackson commented 1 year ago

Thank you for your speedy reply. I realised that how to download weights model now.

However, I am not so clear about the error occured when running the code of ./run.sh bert-optiprompt.

Environment: python3.6

Issues: 


If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForMaskedLM for predictions without further training.
Traceback (most recent call last):
  File "train.py", line 880, in <module>
    main()
  File "train.py", line 724, in main
    load_from_cache_file=not data_args.overwrite_cache,
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1318, in map
    transformed_shards = [r.get() for r in results]
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1318, in <listcomp>
    transformed_shards = [r.get() for r in results]
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get
    raise self._value
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/multiprocess/pool.py", line 424, in _handle_tasks
    put(task)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/multiprocess/connection.py", line 209, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/multiprocess/reduction.py", line 54, in dumps
    cls(buf, protocol, *args, **kwds).dump(obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 498, in dump
    StockPickler.dump(self, obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 990, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 1496, in save_function
    obj.__dict__, fkwdefaults), obj=obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 610, in save_reduce
    save(args)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 1227, in save_cell
    pickler.save_reduce(_create_cell, (f,), obj=obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 610, in save_reduce
    save(args)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 605, in save_reduce
    save(cls)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 1423, in save_type
    obj.__bases__, _dict), obj=obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 610, in save_reduce
    save(args)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 751, in save_tuple
    save(element)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 990, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/dill/_dill.py", line 990, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 507, in save
    self.save_global(obj, rv)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/pickle.py", line 927, in save_global
    (obj, module_name, name))
_pickle.PicklingError: Can't pickle typing.Union[str, NoneType]: it's not the same object as typing.Union
2023-11-01 21:28:09,031 : Starting new HTTPS connection (1): huggingface.co:443
2023-11-01 21:28:09,267 : https://huggingface.co:443 "HEAD /result/bert-optiprompt/resolve/main/config.json HTTP/1.1" 401 0
401 Client Error: Unauthorized for url: https://huggingface.co/result/bert-optiprompt/resolve/main/config.json
Traceback (most recent call last):
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/configuration_utils.py", line 420, in get_config_dict
    use_auth_token=use_auth_token,
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/file_utils.py", line 1056, in cached_path
    local_files_only=local_files_only,
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/file_utils.py", line 1186, in get_from_cache
    r.raise_for_status()
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/result/bert-optiprompt/resolve/main/config.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "evaluation.py", line 423, in <module>
    main()
  File "evaluation.py", line 118, in main
    model = AutoModel.from_pretrained(args.model_name_or_path)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/models/auto/modeling_auto.py", line 724, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/models/auto/configuration_auto.py", line 360, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/userhome/cs2/u3619509/anaconda3/envs/python36/lib/python3.6/site-packages/transformers/configuration_utils.py", line 432, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for 'result/bert-optiprompt'. Make sure that:

- 'result/bert-optiprompt' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'result/bert-optiprompt' is the correct path to a directory containing a config.json file```

Could you help me to solve this problem? 

Thank you for your reply again!
kongds commented 1 year ago

It seems you python version is too old to pickle object for huggingface datasets. We uses python3.8 in promptbert.

Louis-Jackson commented 1 year ago

Thanks a lot! It works now.

Louis-Jackson commented 1 year ago

Sorry, one more question...

When I am running the supervised Prompt-BERT training, several training issues occured and I cannot solve them.

The issues are listed below:

/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launch.py:181: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use-env is set by default in torchrun.
If your script expects `--local-rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  warnings.warn(
[2023-11-03 19:32:19,748] torch.distributed.run: [WARNING]
[2023-11-03 19:32:19,748] torch.distributed.run: [WARNING] *****************************************
[2023-11-03 19:32:19,748] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2023-11-03 19:32:19,748] torch.distributed.run: [WARNING] *****************************************
Traceback (most recent call last):
Traceback (most recent call last):
  File "train.py", line 880, in <module>
  File "train.py", line 880, in <module>
Traceback (most recent call last):
  File "train.py", line 880, in <module>
    main()
  File "train.py", line 480, in main
        main()main()

  File "train.py", line 480, in main
  File "train.py", line 480, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/transformers/hf_argparser.py", line 166, in parse_args_into_dataclasses
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/transformers/hf_argparser.py", line 166, in parse_args_into_dataclasses
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/transformers/hf_argparser.py", line 166, in parse_args_into_dataclasses
            raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")

ValueError: ValueErrorValueErrorSome specified arguments are not used by the HfArgumentParser: ['--local-rank=3']: :
Some specified arguments are not used by the HfArgumentParser: ['--local-rank=2']Some specified arguments are not used by the HfArgumentParser: ['--local-rank=1']

Traceback (most recent call last):
  File "train.py", line 880, in <module>
    main()
  File "train.py", line 480, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/transformers/hf_argparser.py", line 166, in parse_args_into_dataclasses
    raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--local-rank=0']
[2023-11-03 19:32:24,763] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 2640) of binary: /home/louis/anaconda3/envs/Prompt-BERT/bin/python
Traceback (most recent call last):
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launch.py", line 196, in <module>
    main()
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launch.py", line 192, in main
    launch(args)
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launch.py", line 177, in launch
    run(args)
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/louis/anaconda3/envs/Prompt-BERT/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
train.py FAILED

How could I solve them?

Really appreciate your reply.

kongds commented 1 year ago

which PyTorch version do you use? If you use PyTorch 2+, maybe you need use torchrun instead of python -m torch.distributed.launch in this line: https://github.com/kongds/Prompt-BERT/blob/24c9c20f1b3957f7de833ed2479d29488ad6a791/run.sh#L126

Louis-Jackson commented 1 year ago

I have rewritten the old version of the PyTorch training commands, and they are now works correctly.

Due to my unfamiliarity with PyTorch, I apologize for any inconvenience caused.

Thank you very much for your assistance!