Error while running evaluate_linking_results.py on new results file

[agolo-alan-hogue]

Hello again!

I keep seeing errors here and there, but they all look superficially like they might be easy fixes. I hope so, I was really hoping to have some experiments run and showing up in the UI in the next few days. I really appreciate any help you can offer.

In a previous issue I reported an error running an experiment with REL.

I tried doing the same with REfined, and it finished running the linking with no errors. So then I ran the evaluation script and got the following. The name of that file (./data/wikidata_mappings/alias_to_qids.db.back) looks suspicious...could it be a simple error in the name of the file it's looking for? I have checked and I DO have ./data/wikidata_mappings/alias_to_qids.db.

python3 evaluate_linking_results.py evaluation-results/refined/refined.agolo-e2e-eval.linked_articles.jsonl
2023-11-06 14:53:49 [INFO]: Evaluating linking results from ['evaluation-results/refined/refined.agolo-e2e-eval.linked_articles.jsonl'] ...
2023-11-06 14:53:49 [INFO]: Loading whitelist types from small-data-files/whitelist_types.tsv ...
2023-11-06 14:53:49 [INFO]: Loading whitelist type adjustments from small-data-files/type_adjustments.txt ...
2023-11-06 14:53:49 [INFO]: -> Whitelist type adjustments loaded.
2023-11-06 14:53:49 [INFO]: -> 29 whitelist types loaded.
2023-11-06 14:53:49 [INFO]: Initializing entity database for evaluation ...
2023-11-06 14:53:49 [INFO]: Loading entity ID to name database from ./data/wikidata_mappings/qid_to_label.db ...
2023-11-06 14:55:06 [INFO]: -> 86599035 entity ID to name mappings loaded.
2023-11-06 14:55:06 [INFO]: Loading entity ID to whitelist types database from ./data/wikidata_mappings/qid_to_whitelist_types.db ...
2023-11-06 14:56:14 [INFO]: -> 77316268 entity ID to whitelist types mappings loaded.
2023-11-06 14:56:14 [INFO]: Loading whitelist type adjustments from small-data-files/type_adjustments.txt ...
2023-11-06 14:56:14 [INFO]: -> Whitelist type adjustments loaded.
2023-11-06 14:56:14 [INFO]: Loading Wikipedia to Wikidata database from ./data/wikidata_mappings/wikipedia_name_to_qid.db ...
2023-11-06 14:56:17 [INFO]: -> 9279408 Wikipedia-Wikidata mappings loaded.
2023-11-06 14:56:17 [INFO]: Loading redirects database from ./data/wikipedia_mappings/redirects.db ...
2023-11-06 14:56:21 [INFO]: -> 10914101 redirects loaded.
2023-11-06 14:56:21 [INFO]: Loading demonyms from ./data/wikidata_mappings/qid_to_demonym.tsv ...
2023-11-06 14:56:21 [INFO]: -> 1646 demonyms loaded
2023-11-06 14:56:21 [INFO]: Loading real numbers from ./data/wikidata_mappings/quantity.tsv ...
2023-11-06 14:56:21 [INFO]: -> 136867 real numbers loaded.
2023-11-06 14:56:21 [INFO]: Loading points in time from ./data/wikidata_mappings/datetime.tsv ...
2023-11-06 14:56:21 [INFO]: -> 194619 points in time loaded.
2023-11-06 14:56:21 [INFO]: Loading family name aliases into entity database ...
2023-11-06 14:56:21 [INFO]: Yielding given name mapping from ./data/wikidata_mappings/qid_to_name.tsv ...
2023-11-06 14:56:31 [INFO]: -> 1573195 family name aliases loaded into entity database.
2023-11-06 14:56:31 [INFO]: Loading alias to entity IDs database from ./data/wikidata_mappings/alias_to_qids.db.back ...
Traceback (most recent call last):
  File "/Users/alan/repos/agolo/elevant/evaluate_linking_results.py", line 178, in <module>
    main(cmdl_args)
  File "/Users/alan/repos/agolo/elevant/evaluate_linking_results.py", line 43, in main
    evaluator = Evaluator(type_mapping_file, whitelist_file=whitelist_file, contains_unknowns=not args.no_unknowns,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alan/repos/agolo/elevant/src/evaluation/evaluator.py", line 81, in __init__
    self.entity_db = load_evaluation_entities(type_mapping_file, custom_kb)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alan/repos/agolo/elevant/src/evaluation/evaluator.py", line 30, in load_evaluation_entities
    entity_db.load_alias_to_entities()
  File "/Users/alan/repos/agolo/elevant/src/models/entity_database.py", line 204, in load_alias_to_entities
    self.alias_to_entities_db = EntityDatabaseReader.get_alias_to_entities_db()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alan/repos/agolo/elevant/src/helpers/entity_database_reader.py", line 278, in get_alias_to_entities_db
    aliases_db = EntityDatabaseReader.read_from_dbm(filename, value_type=set)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alan/repos/agolo/elevant/src/helpers/entity_database_reader.py", line 225, in read_from_dbm
    dbm_db = dbm.open(db_file, "r")
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dbm/__init__.py", line 85, in open
    raise error[0]("db file doesn't exist; "
dbm.error: db file doesn't exist; use 'c' or 'n' flag to create a new db

[flackbash]

Yes, alias_to_qids.db.back should not exist and it should not try to read from such a file. It should be alias_to_qids.db. The file paths are specified here. Did you make any changes to that file by any chance?

[agolo-alan-hogue]

I actually think I did, possibly, while downloading the data by hand. It looks like this has cleared it up. Thank you!