search

RTXteam / RTX-KG2

Build system for the RTX-KG2 biomedical knowledge graph, part of the ARAX reasoning system (https://github.com/RTXTeam/RTX)
MIT License
39 stars 8 forks source link

UMLS Conversion Script (Multi Ont) Fails #300

Closed ecwood closed 1 year ago

ecwood commented 1 year ago

While testing for https://github.com/RTXteam/RTX-KG2/issues/296, I found that multi_ont_to_json_kg.py fails:

+ set -o nounset -o pipefail -o errexit
+ [[ /home/ubuntu/kg2-build/umls_cuis.tsv == \-\-\h\e\l\p ]]
+ [[ /home/ubuntu/kg2-build/umls_cuis.tsv == \-\h ]]
+ echo '================= starting build-multi-ont-kg.sh ================='
================= starting build-multi-ont-kg.sh =================
+ date
Mon Jun 26 18:30:26 UTC 2023
++ dirname build-multi-ont-kg.sh
+ config_dir=.
+ source ./master-config.shinc
++ '[' -z ']'
++ test_suffix=
++ BUILD_DIR=/home/ubuntu/kg2-build
++ VENV_DIR=/home/ubuntu/kg2-venv
++ CODE_DIR=/home/ubuntu/kg2-code
++ umls_dir=/home/ubuntu/kg2-build/umls
++ umls_dest_dir=/home/ubuntu/kg2-build/umls/META
++ s3_region=us-west-2
++ s3_bucket=rtx-kg2
++ s3_bucket_public=rtx-kg2-public
++ s3_bucket_versioned=rtx-kg2-versioned
++ s3_cp_cmd='aws s3 cp --no-progress --region us-west-2'
++ mysql_conf=/home/ubuntu/kg2-build/mysql-config.conf
++ curl_get='curl -s -L -f'
++ curies_to_categories_file=/home/ubuntu/kg2-code/curies-to-categories.yaml
++ curies_to_urls_file=/home/ubuntu/kg2-code/curies-to-urls-map.yaml
++ predicate_mapping_file=/home/ubuntu/kg2-code/predicate-remap.yaml
++ infores_mapping_file=/home/ubuntu/kg2-code/kg2-provided-by-curie-to-infores-curie.yaml
++ ont_load_inventory_file=/home/ubuntu/kg2-code/ont-load-inventory.yaml
++ umls2rdf_config_master=/home/ubuntu/kg2-code/umls2rdf-umls.conf
++ rtx_config_file=RTXConfiguration-config.json
++ biolink_model_version=3.1.2
+ build_flag=
+ biolink_base_url_no_version=https://raw.githubusercontent.com/biolink/biolink-model/
+ biolink_raw_base_url=https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/biolink-model.owl.ttl
+ biolink_raw_base_url_curies_urls_map=https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/
+ curies_urls_map_replace_string='\    biolink_download_source: https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/'
+ ont_load_inventory_replace_string='\  url: https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/biolink-model.owl.ttl'
+ sed -i '\@https://raw.githubusercontent.com/biolink/biolink-model/@c\  url: https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/biolink-model.owl.ttl' /home/ubuntu/kg2-code/ont-load-inventory.yaml
+ sed -i '\@https://raw.githubusercontent.com/biolink/biolink-model/@c\    biolink_download_source: https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/' /home/ubuntu/kg2-code/curies-to-urls-map.yaml
+ [[ '' == \t\e\s\t ]]
+ [[ '' == \a\l\l\t\e\s\t ]]
+ test_suffix=
+ test_arg=
+ umls_cuis_file=/home/ubuntu/kg2-build/umls_cuis.tsv
+ output_file=/home/ubuntu/kg2-build/kg2-ont.json
++ basename /home/ubuntu/kg2-build/kg2-ont.json
+ output_file_base=kg2-ont.json
++ dirname /home/ubuntu/kg2-build/kg2-ont.json
+ log_file=/home/ubuntu/kg2-build/build-kg2-ont-stderr.log
+ output_file_base=/home/ubuntu/kg2-build/kg2-ont
+ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ubuntu/kg2-build
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ubuntu/kg2-build
++ /home/ubuntu/kg2-code/get-system-memory-gb.sh
+ mem_gb=249
+ export OWLTOOLS_MEMORY=249G
+ OWLTOOLS_MEMORY=249G
+ export DEBUG=1
+ DEBUG=1
+ node_datatype_properties_file=/home/ubuntu/kg2-build/node_datatype_properties.json
+ /home/ubuntu/kg2-build/robot convert --input /home/ubuntu/kg2-build/umls-hgnc.ttl --output /home/ubuntu/kg2-build/umls-hgnc.owl
+ /home/ubuntu/kg2-build/robot convert --input /home/ubuntu/kg2-build/umls-omim.ttl --output /home/ubuntu/kg2-build/umls-omim.owl
+ /home/ubuntu/kg2-venv/bin/python3 -u /home/ubuntu/kg2-code/save_owl_datatypeproperties.py /home/ubuntu/kg2-build/umls-hgnc.owl /home/ubuntu/kg2-build/umls-omim.owl --outputFile /home/ubuntu/kg2-build/node_datatype_properties.json
+ cd /home/ubuntu/kg2-build
+ /home/ubuntu/kg2-venv/bin/python3 -u /home/ubuntu/kg2-code/multi_ont_to_json_kg.py /home/ubuntu/kg2-code/curies-to-categories.yaml /home/ubuntu/kg2-code/curies-to-urls-map.yaml /home/ubuntu/kg2-code/ont-load-inventory.yaml /home/ubuntu/kg2-build/kg2-ont.json /home/ubuntu/kg2-build/umls_cuis.tsv /home/ubuntu/kg2-build/node_datatype_properties.json
/home/ubuntu/kg2-venv/lib/python3.7/site-packages/rdflib_jsonld/__init__.py:12: DeprecationWarning: The rdflib-jsonld package has been integrated into rdflib as of rdflib==6.0.0.  Please remove rdflib-jsonld from your project's dependencies.
  DeprecationWarning,
https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/biolink-model.owl.ttl
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 139, in make_kg2
    ont_source_info_dict['file'])
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 635, in download_file_if_not_exist_locally
    with urllib.request.urlopen(url, context=ctx) as u, open(temp_file_name, 'wb') as f:
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
ecwood commented 1 year ago

This link (https://raw.githubusercontent.com/biolink/biolink-model/3.1.2/biolink-model.owl.ttl) is no longer valid. Also, @acevedol and @saramsey, is it correct that we are on Biolink version 3.1.2? This is what the issue-263 and issue-263-local branch have in their master-config.shinc files (linked) as well.

ecwood commented 1 year ago

With 4358a2f and 23e1f4b, the Biolink model URL resolves. However, these errors (far more than these listed, but these are an example) are popping up:

2023-06-26 20:26:52,680 ERROR org.semanticweb.owlapi.rdf.rdfxml.parser.OWLRDFConsumer - Entity not properly recognized, missing triples in input? http://org.semanticweb.owlapi/error#Error1 for type Class
2023-06-26 20:26:52,787 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/start_interbase_coordinate> "end interbase coordinate"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,788 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/enabled_by> "prevented by"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,790 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/prevents> "predisposes"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,792 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/decreases_response_to> "increases response to"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,792 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/broad_match> "narrow match"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,792 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/enables> "prevents"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,793 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_increased_amount> "has decreased amount"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,793 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_output> "has input"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,794 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/predisposes> "prevents"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,794 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/increases_response_to> "decreases response to"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,794 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/contraindicated_for> "treats"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,795 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_input> "has output"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,795 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/disrupts> "enables"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,796 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/positively_correlated_with> "negatively correlated with"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,796 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_not_completed> "has completed"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,796 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/ameliorates> "exacerbates"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,797 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/lacks_part> "has part"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,798 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_active_ingredient> "is excipient of"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,798 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/causes> "prevents"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,798 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_part> "lacks part"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,799 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/negatively_correlated_with> "positively correlated with"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,799 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_completed> "has not completed"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,800 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/narrow_match> "broad match"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,800 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/has_decreased_amount> "has increased amount"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')
2023-06-26 20:26:52,801 ERROR org.semanticweb.owlapi.util.OWLAnnotationPropertyTransformer - Attempt to transform an axiom to correct misuse of properties failed. Property replacement: {<http://purl.org/pav/version>=<http://purl.org/pav/version>, <https://w3id.org/biolink/vocab/opposite_of>=<https://w3id.org/biolink/vocab/opposite_of>, rdfs:label=rdfs:label}, axiom: AnnotationAssertion(<https://w3id.org/biolink/vocab/opposite_of> <https://w3id.org/biolink/vocab/prevented_by> "enabled by"), error: class uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression cannot be cast to class org.semanticweb.owlapi.model.IRI (uk.ac.manchester.cs.owl.owlapi.OWLLiteralImplNoCompression and org.semanticweb.owlapi.model.IRI are in unnamed module of loader 'app')

These don't cause script failure, but they seem undesireable.

ecwood commented 1 year ago

Here's the new script failing error, though:

Reading ontology file: umls-hcpcs.ttl; size: 10035.03 KiB
Reading ontology JSON file: umls-hcpcs.json; size: 7530.39 KiB
file: umls-hcpcs.ttl; last modified: 2023-06-23 20:33:55 GMT
Reading ontology file: umls-hgnc.ttl; size: 89992.21 KiB
Reading ontology JSON file: umls-hgnc.json; size: 55320.90 KiB
file: umls-hgnc.ttl; last modified: 2023-06-23 20:34:23 GMT
Reading ontology file: umls-hl7.ttl; size: 8821.98 KiB
Reading ontology JSON file: umls-hl7.json; size: 10367.00 KiB
file: umls-hl7.ttl; last modified: 2023-06-23 20:34:25 GMT
Reading ontology file: umls-hpo.ttl; size: 13904.69 KiB
Reading ontology JSON file: umls-hpo.json; size: 23080.24 KiB
file: umls-hpo.ttl; last modified: 2023-06-23 20:34:29 GMT
Reading ontology file: umls-icd10pcs.ttl; size: 103108.02 KiB
Reading ontology JSON file: umls-icd10pcs.json; size: 209355.56 KiB
file: umls-icd10pcs.ttl; last modified: 2023-06-23 20:34:58 GMT
Reading ontology file: umls-icd9cm.ttl; size: 10464.25 KiB
Reading ontology JSON file: umls-icd9cm.json; size: 21380.25 KiB
file: umls-icd9cm.ttl; last modified: 2023-06-23 20:35:01 GMT
Reading ontology file: umls-lnc.ttl; size: 698076.19 KiB
-1239455854
Use the -vvv option to show the stack trace.
Use the --help option to see usage information.
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 147, in make_kg2
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 71, in load_ont_file_return_ontology_and_metadata
    ontology = kg2_util.make_ontology_from_local_file(file_name, save_pickle=save_pickle)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 821, in make_ontology_from_local_file
    assert cp.returncode == 0
AssertionError
ecwood commented 1 year ago

New failure point:

file: ro.owl; last modified: 2023-06-27 00:02:38 GMT
http://purl.obolibrary.org/obo/uberon/ext.owl
Reading ontology file: uberon-ext.owl; size: 87307.03 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface uberon-ext.owl -o -f json /tmp/kg2-dpp9nnio.json
Reading ontology JSON file: uberon-ext.json; size: 48357.57 KiB
file: uberon-ext.owl; last modified: 2023-06-27 00:02:44 GMT
http://www.ebi.ac.uk/efo/efo.owl
Reading ontology file: efo.owl; size: 171246.86 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface efo.owl -o -f json /tmp/kg2-2l2jznsr.json
Reading ontology JSON file: efo.json; size: 66358.45 KiB
file: efo.owl; last modified: 2023-06-27 00:04:25 GMT
http://purl.obolibrary.org/obo/fma.owl
Reading ontology file: fma.owl; size: 64812.54 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface fma.owl -o -f json /tmp/kg2-d96ojbx8.json
Reading ontology JSON file: fma.json; size: 51716.15 KiB
file: fma.owl; last modified: 2023-06-27 00:05:54 GMT
http://purl.obolibrary.org/obo/ddanat.owl
Reading ontology file: ddanat.owl; size: 294.65 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface ddanat.owl -o -f json /tmp/kg2-rqhdx98l.json
Reading ontology JSON file: ddanat.json; size: 163.82 KiB
file: ddanat.owl; last modified: 2023-06-27 00:07:29 GMT
http://purl.obolibrary.org/obo/cl.owl
Reading ontology file: cl.owl; size: 53896.60 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface cl.owl -o -f json /tmp/kg2-o8smx0tg.json
Reading ontology JSON file: cl.json; size: 31005.87 KiB
file: cl.owl; last modified: 2023-06-27 00:07:36 GMT
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 142, in make_kg2
    assert os.path.exists(ont_source_info_dict['file']), local_file_name
AssertionError: foodon.owl

This relates to #236.

ecwood commented 1 year ago

After attempting to fix this by reverting back to download: true, I encountered this error:

Reading ontology JSON file: fma.json; size: 51716.15 KiB
file: fma.owl; last modified: 2023-06-27 00:05:54 GMT
http://purl.obolibrary.org/obo/ddanat.owl
Reading ontology file: ddanat.owl; size: 294.65 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface ddanat.owl -o -f json /tmp/kg2-2t9kd04z.json
Reading ontology JSON file: ddanat.json; size: 163.82 KiB
file: ddanat.owl; last modified: 2023-06-27 00:07:29 GMT
http://purl.obolibrary.org/obo/cl.owl
Reading ontology file: cl.owl; size: 53896.60 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface cl.owl -o -f json /tmp/kg2-6yh65aok.json
Reading ontology JSON file: cl.json; size: 31005.87 KiB
file: cl.owl; last modified: 2023-06-27 00:07:36 GMT
http://purl.obolibrary.org/obo/foodon.owl
Reading ontology file: foodon.owl; size: 36605.22 KiB
/usr/bin/java -Xms2G -Xmx249G -DentityExpansionLimit=4086000 -Djava.awt.headless=true -classpath /home/ubuntu/kg2-build/owltools owltools.cli.CommandLineInterface foodon.owl -o -f json /tmp/kg2-5cqcj95p.json
Exception in thread "main" java.lang.ClassCastException: class uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl cannot be cast to class org.semanticweb.owlapi.model.OWLObjectProperty (uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl and org.semanticweb.owlapi.model.OWLObjectProperty are in unnamed module of loader 'app')
    at org.geneontology.obographs.owlapi.FromOwl.getDRBuilder(FromOwl.java:597)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraph(FromOwl.java:212)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraphDocument(FromOwl.java:112)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:264)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:209)
    at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:3712)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
    at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 147, in make_kg2
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 71, in load_ont_file_return_ontology_and_metadata
    ontology = kg2_util.make_ontology_from_local_file(file_name, save_pickle=save_pickle)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 813, in make_ontology_from_local_file
    check=True)
  File "/usr/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['owltools', 'foodon.owl', '-o', '-f', 'json', '/tmp/kg2-5cqcj95p.json']' returned non-zero exit status 1.
ecwood commented 1 year ago

Here is the error that command returns:

ubuntu@ip-172-31-59-112:~/kg2-build$ ./owltools foodon.owl -o -f json /tmp/kg2-5cqcj95p.json
Exception in thread "main" java.lang.ClassCastException: class uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl cannot be cast to class org.semanticweb.owlapi.model.OWLObjectProperty (uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl and org.semanticweb.owlapi.model.OWLObjectProperty are in unnamed module of loader 'app')
    at org.geneontology.obographs.owlapi.FromOwl.getDRBuilder(FromOwl.java:597)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraph(FromOwl.java:212)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraphDocument(FromOwl.java:112)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:264)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:209)
    at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:3712)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
    at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)

(I am confident the command is correct because it completes successfully on bfo.owl which completes in multi_ont_to_json_kg.py).

ecwood commented 1 year ago

The fix used in #236 isn't going to work again, since the imports folder has been removed from the master branch of the Food Ontology repository (https://github.com/FoodOntology/foodon):

ubuntu@ip-172-31-59-112:~/kg2-build/foodon/foodon-0.5.1$ ~/kg2-build/owltools foodon.owl -o -f json /tmp/kg2-5cqcj95p.json
2023-06-27 20:54:18,242 ERROR (CommandRunner:4815) could not parse:foodon.owl
org.semanticweb.owlapi.model.UnloadableImportException: Could not load imported ontology: <http://purl.obolibrary.org/obo/foodon/imports/bfo_import.owl> Cause: https://raw.githubusercontent.com/FoodOntology/foodon/master/imports/bfo_import.owl
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1870)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.TripleHandlers$TPImportsHandler.handleTriple(TripleHandlers.java:1537)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.TripleHandlers$HandlerAccessor.handleStreaming(TripleHandlers.java:194)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.OWLRDFConsumer.statementWithResourceValue(OWLRDFConsumer.java:1545)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFParser.statementWithResourceValue(RDFParser.java:370)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.EmptyPropertyElement.startElement(StartRDF.java:236)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.PropertyElementList.startElement(StartRDF.java:658)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFParser.startElement(RDFParser.java:201)
    at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
    at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
    at org.apache.xerces.impl.dtd.XMLDTDValidator.emptyElement(Unknown Source)
    at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFParser.parse(RDFParser.java:145)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:73)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:220)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1254)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1208)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1108)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1064)
    at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:163)
    at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:150)
    at owltools.io.ParserWrapper.parse(ParserWrapper.java:132)
    at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:4803)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
    at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Caused by: org.semanticweb.owlapi.io.OWLOntologyCreationIOException: https://raw.githubusercontent.com/FoodOntology/foodon/master/imports/bfo_import.owl
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:230)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1254)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1208)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1108)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadImports(OWLOntologyManagerImpl.java:1825)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1863)
    ... 33 more
Caused by: java.io.FileNotFoundException: https://raw.githubusercontent.com/FoodOntology/foodon/master/imports/bfo_import.owl
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1974)
    at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1969)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1968)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
    at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStreamFromContentEncoding(AbstractOWLParser.java:179)
    at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:141)
    at org.semanticweb.owlapi.io.AbstractOWLParser.getInputSource(AbstractOWLParser.java:264)
    at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:72)
    at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:220)
    ... 38 more
Caused by: java.io.FileNotFoundException: https://raw.githubusercontent.com/FoodOntology/foodon/master/imports/bfo_import.owl
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1920)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3118)
    at java.base/java.net.URLConnection.getContentEncoding(URLConnection.java:530)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentEncoding(HttpsURLConnectionImpl.java:406)
    at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:136)
    ... 41 more

I think this is related to the Food Ontology's Issue Number 275.

ecwood commented 1 year ago

Here is documentation on uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl

Here is documentation on org.semanticweb.owlapi.model.OWLObjectProperty

Here is a discussion of the same error message. It sounds like the problem is that they are not part of the same hierarchy, so one can't be cast to the other. My next step is going to be to try and track down where these different properties show up in the source file so that I can report it to the Food Ontology to be fixed.

ecwood commented 1 year ago

Based on #303, I switched to a pickle file for FOODON. There is a new error though:

Ont: https://w3id.org/biolink/biolink-model.owl.ttl handle: biolink-model.owl.json meta: {'subsets': [], 'xrefs': [], 'basicPropertyValues': [{'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/Annotation'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/CaseToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/CellLineToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ChemicalEntityOrGeneOrGeneProduct'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ChemicalEntityOrProteinOrPolypeptide'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ChemicalEntityToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ChemicalOrDrugOrTreatment'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/DiseaseOrPhenotypicFeatureToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/DiseaseToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/Entity'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/EntityToDiseaseOrPhenotypicFeatureAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/EntityToExposureEventAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/EntityToOutcomeAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/EpigenomicEntity'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/FrequencyQualifierMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/GeneExpressionMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/GeneGroupingMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/GeneToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/GenomicEntity'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/GenotypeToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/MacromolecularMachineMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/MacromolecularMachineToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/MappingCollection'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/MaterialSampleToEntityAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ModelToDiseaseAssociationMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/OntologyClass'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/OrganismTaxonToEntityAssociation'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/Outcome'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/PathologicalEntityMixin'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/PhysicalEssenceOrOccurrent'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/PredicateMapping'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/RelationshipQuantifier'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/SubjectOfInvestigation'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/ThingWithTaxon'}, {'pred': 'http://purl.obolibrary.org/obo/IAO_0000700', 'val': 'https://w3id.org/biolink/vocab/VariantToEntityAssociationMixin'}, {'pred': 'http://purl.org/dc/terms/license', 'val': 'https://creativecommons.org/publicdomain/zero/1.0/'}, {'pred': 'http://purl.org/pav/version', 'val': '3.1.2'}, {'pred': 'http://www.w3.org/2000/01/rdf-schema#label', 'val': 'Biolink-Model'}, {'pred': 'https://w3id.org/linkml/generation_date', 'val': '2022-12-20T18:46:15'}, {'pred': 'https://w3id.org/linkml/metamodel_version', 'val': '1.7.0'}, {'pred': 'https://w3id.org/linkml/source_file', 'val': 'biolink-model.yaml'}, {'pred': 'https://w3id.org/linkml/source_file_date', 'val': '2022-12-20T18:45:00'}, {'pred': 'https://w3id.org/linkml/source_file_size', 'val': '334572'}]}
Base URL: https://w3id.org/biolink/biolink-model https://w3id.org/biolink/biolink-model.owl.ttl
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 181, in make_kg2
    select_datatype_properties) # temporary addition for Ontobio Issue #507
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 530, in make_nodes_dict_from_ontologies_list
    assert first_ontology.id == kg2_util.BASE_URL_BIOLINK_ONTOLOGY, "biolink needs to be first in ont-load-inventory.yaml"
AssertionError: biolink needs to be first in ont-load-inventory.yaml

This is weird because biolink is first in the ont-load-inventory.yaml file.

I think this is because of https://github.com/RTXteam/RTX-KG2/commit/4f7a1e21453591bb4ff3c686f1200a01e03b345f#diff-3a03aca522a201f93e621bc75bdea21a1e34525350f60392470aaf10a5bbdcebL130-R130

saramsey commented 1 year ago

In the interest of minimizing pain in future builds, I wonder if we should loosen up that assertion to just search for substring "biolink-model" in the URL? The Biolink w3id URL seems to change from time to time.

ecwood commented 1 year ago

That is probably a good idea. I corrected it enough (in kg2_util.py) to continue the testing with db8449c, but we should find a more sustainable solution.

I also want to note my concern with the current state of build-multi-ont-kg.sh. It has been processing LOINC for 16 hours.

[https://raw.githubusercontent.com/biolink/biolink-model/v3.1.2/biolink-model.owl.ttl] Unable to obtain a CURIE for ontology node ID: https://github.com/cmungall/owlstar/blob/master/owlstar.ttlSomeSomeInterpretation
[https://w3id.org/biolink/biolink-model.owl.ttl] Unable to expand ontology node ID to an IRI: pav:version
[https://raw.githubusercontent.com/biolink/biolink-model/v3.1.2/biolink-model.owl.ttl] Unable to obtain a CURIE for ontology node ID: pav:version
[https://w3id.org/biolink/biolink-model.owl.ttl] could not shorten this IRI to a CURIE: http://org.semanticweb.owlapi/error#Error1
[https://raw.githubusercontent.com/biolink/biolink-model/v3.1.2/biolink-model.owl.ttl] Unable to obtain a CURIE for ontology node ID: http://org.semanticweb.owlapi/error#Error1
processing ontology: STY: start 2023-06-29 01:37:03.691095
processing ontology: umls_source:ATC start 2023-06-29 01:37:03.814896
processing ontology: umls_source:DRUGBANK start 2023-06-29 01:37:23.324602
processing ontology: umls_source:FMA start 2023-06-29 01:37:51.638118
processing ontology: umls_source:GO start 2023-06-29 01:45:05.972972
processing ontology: umls_source:HCPCS start 2023-06-29 02:03:05.046571
processing ontology: umls_source:HGNC start 2023-06-29 02:03:17.549618
processing ontology: umls_source:HL7 start 2023-06-29 02:05:40.581005
processing ontology: umls_source:HPO start 2023-06-29 02:06:06.428024
processing ontology: umls_source:ICD10PCS start 2023-06-29 02:07:24.653163
processing ontology: umls_source:ICD9CM start 2023-06-29 02:16:59.932934
processing ontology: umls_source:LNC start 2023-06-29 02:18:00.542861
ubuntu@ip-172-31-59-112:~/kg2-build$ date
Thu Jun 29 18:50:49 UTC 2023
ecwood commented 1 year ago

Nevermind my concern with LOINC. Based on the previous build on buildkg2.rtx.ai, it takes around 27 hours:

processing ontology: umls_source:LNC start 2023-05-20 03:30:08.252845
processing ontology: umls_source:MED-RT start 2023-05-21 06:00:47.781519
saramsey commented 1 year ago

That leaves me to wonder, is LOINC worth it? I will ask around.

ecwood commented 1 year ago

New error:

Calling make_map_of_node_ontology_ids_to_curie_ids
Calling get_rels_dict
[https://w3id.org/biolink/biolink-model.owl.ttl] subject node ontology ID has no curie ID in the map: pav:version
[https://w3id.org/biolink/biolink-model.owl.ttl] subject node ontology ID has no curie ID in the map: http://schema.org/logo
[https://w3id.org/biolink/biolink-model.owl.ttl] object node ontology ID has no curie ID in the map: http://org.semanticweb.owlapi/error#Error1
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 194, in make_kg2
    map_of_node_ontology_ids_to_curie_ids)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1139, in get_rels_dict
    key = make_rel_key(node_id, CURIE_OBO_XREF, xref_node_id, provided_by)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 351, in make_rel_key
    key += ';' + ontology_id
TypeError: can only concatenate str (not "list") to str
ecwood commented 1 year ago

9846346 addressed that previous error. There is a new error:

Calling get_rels_dict
[https://w3id.org/biolink/biolink-model.owl.ttl] subject node ontology ID has no curie ID in the map: pav:version
[https://w3id.org/biolink/biolink-model.owl.ttl] subject node ontology ID has no curie ID in the map: http://schema.org/logo
[https://w3id.org/biolink/biolink-model.owl.ttl] object node ontology ID has no curie ID in the map: http://org.semanticweb.owlapi/error#Error1
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1372, in <module>
    save_pickle)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 194, in make_kg2
    map_of_node_ontology_ids_to_curie_ids)
  File "/home/ubuntu/kg2-code/multi_ont_to_json_kg.py", line 1146, in get_rels_dict
    ontology_update_date)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 735, in make_edge
    edge_id = make_edge_key(edge)
  File "/home/ubuntu/RTX-KG2/kg2_util.py", line 712, in make_edge_key
    edge_dict['primary_knowledge_source']
TypeError: can only concatenate str (not "list") to str

I suspect we need to address a larger, upstream issue that is creating these lists.

saramsey commented 1 year ago

Erica, using the current version (2020-04-06) of owltools downloaded from the github:owlcollab/owltools/releases, and the May 3 version of foodon.owl, I get the same stack trace error:

./owltools foodon.owl -o -f json foodon.json
Exception in thread "main" java.lang.ClassCastException: class uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl cannot be cast to class org.semanticweb.owlapi.model.OWLObjectProperty (uk.ac.manchester.cs.owl.owlapi.OWLObjectInverseOfImpl and org.semanticweb.owlapi.model.OWLObjectProperty are in unnamed module of loader 'app')
    at org.geneontology.obographs.owlapi.FromOwl.getDRBuilder(FromOwl.java:597)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraph(FromOwl.java:212)
    at org.geneontology.obographs.owlapi.FromOwl.generateGraphDocument(FromOwl.java:112)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:264)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:209)
    at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:3712)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
    at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
ecwood commented 1 year ago

I am closing this issue because the code worked in KG2.8.4pre's build.