search

DFKI-NLP / fewie

Few-shot named entity recognition
MIT License
11 stars 1 forks source link

initial "python evaluate.py --help" fails #1

Closed ArneBinder closed 3 years ago

ArneBinder commented 3 years ago

When the project is intially set up (git clone, create environment, pip install), the command python evaluate.py --help fails with:

Traceback (most recent call last):
  File "/mnt/DATA/DEVELOPING/dfki/lenovo/fewie/evaluate.py", line 26, in <module>
    evaluate()
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/main.py", line 32, in decorated_main
    _run_hydra(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/utils.py", line 327, in _run_hydra
    hydra.app_help(config_name=config_name, args_parser=args_parser, args=args)
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 328, in app_help
    cfg = self.compose_config(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 507, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 151, in load_configuration
    return self._load_configuration(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 256, in _load_configuration
    cfg = self._merge_defaults_into_config(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 805, in _merge_defaults_into_config
    hydra_cfg = merge_defaults_list_into_config(hydra_cfg, user_list)
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 777, in merge_defaults_list_into_config
    merged_cfg = self._merge_config(
  File "/home/arne/miniconda3/envs/fewie/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 715, in _merge_config
    raise MissingConfigException(msg, new_cfg, options)
hydra.errors.MissingConfigException: Could not load dataset_processor/transformers.
Available options:
    bert
    spanbert
    transformer

Interestingly, the command works if another one like python evaluate.py dataset=conll2003 dataset_processor=bert encoder=bert evaluation/dataset=nway_kshot_5_1 was successfully called at least once.

chen-yuxuan commented 3 years ago

Fixed in the PR #3.

Because for random encoder, a non-learnable (weight-fixed) linear embedding layer is used which only requires tokenizer from a transformer to provide tokenization as its input. And dataset_processor: transformer does not provide a valid tokenizer compared to all the other choices.

Btw, to make it consistent when it comes to the results from random, always set bert (BertTokenizer) as the tokenizer throughout our experiments.