RTXteam / RTX-KG2

Build system for the RTX-KG2 biomedical knowledge graph, part of the ARAX reasoning system (https://github.com/RTXTeam/RTX)
MIT License
38 stars 8 forks source link

`setup-kg2-build.sh` Fails to Install `pip` #327

Closed ecwood closed 1 year ago

ecwood commented 1 year ago

While working on #312, aws configure was not triggered by setup-kg2-build.sh.

ubuntu@ip-172-31-62-73:~/RTX-KG2$ bash -x setup-kg2-build.sh 
+ set -o nounset -o pipefail -o errexit
+ [[ '' == \-\-\h\e\l\p ]]
+ [[ '' == \-\h ]]
+ build_flag=
++ dirname setup-kg2-build.sh
+ config_dir=.
+ [[ '' == \c\i ]]
+ source ./master-config.shinc
++ '[' -z ']'
++ test_suffix=
++ BUILD_DIR=/home/ubuntu/kg2-build
++ VENV_DIR=/home/ubuntu/kg2-venv
++ CODE_DIR=/home/ubuntu/kg2-code
++ umls_dir=/home/ubuntu/kg2-build/umls
++ umls_dest_dir=/home/ubuntu/kg2-build/umls/META
++ s3_region=us-west-2
++ s3_bucket=rtx-kg2
++ s3_bucket_public=rtx-kg2-public
++ s3_bucket_versioned=rtx-kg2-versioned
++ s3_cp_cmd='aws s3 cp --no-progress --region us-west-2'
++ mysql_conf=/home/ubuntu/kg2-build/mysql-config.conf
++ curl_get='curl -s -L -f'
++ curies_to_categories_file=/home/ubuntu/kg2-code/curies-to-categories.yaml
++ curies_to_urls_file=/home/ubuntu/kg2-code/curies-to-urls-map.yaml
++ predicate_mapping_file=/home/ubuntu/kg2-code/predicate-remap.yaml
++ infores_mapping_file=/home/ubuntu/kg2-code/kg2-provided-by-curie-to-infores-curie.yaml
++ ont_load_inventory_file=/home/ubuntu/kg2-code/ont-load-inventory.yaml
++ umls2rdf_config_master=/home/ubuntu/kg2-code/umls2rdf-umls.conf
++ rtx_config_file=RTXConfiguration-config.json
++ biolink_model_version=3.5.1
+ [[ '' != \t\e\s\t ]]
+ test_str=
+ mysql_user=ubuntu
+ mysql_password=1337
+ [[ '' != \c\i ]]
+ psql_user=ubuntu
+ mkdir -p /home/ubuntu/kg2-build
+ setup_log_file=/home/ubuntu/kg2-build/setup-kg2-build.log
+ touch /home/ubuntu/kg2-build/setup-kg2-build.log
+ [[ '' == \c\i ]]
ecwood commented 1 year ago

This was the actual line that failed:

++ curl -s https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py
ecwood commented 1 year ago

Here's the error:

ubuntu@ip-172-31-62-73:~/kg2-code$ curl https://bootstrap.pypa.io/get-pip.py
curl: (60) SSL: no alternative certificate subject name matches target host name 'bootstrap.pypa.io'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
ecwood commented 1 year ago

This appears to be a known issue: https://github.com/pypa/get-pip/issues/178

ecwood commented 1 year ago

Now, the script fails here:

curl -s -L -f -H 'Accept: application/zip' https://github.com/RTXteam/robot/releases/download/v1.3.0/robot.jar
ecwood commented 1 year ago

Honestly, I think the problem is curl on this instance, which seems to be marking every webpage as not having a certificate:

ubuntu@ip-172-31-62-73:~/RTX-KG2$ curl -L -f https://raw.githubusercontent.com/biolink/biolink-model/master/semmed-exclude-list.yaml
curl: (60) SSL: no alternative certificate subject name matches target host name 'raw.githubusercontent.com'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
ecwood commented 1 year ago

This is surprising because kg2erica.rtx.ai and buildkg2.rtx.ai (where the problem is) both have the same version of curl:

kg2erica.rtx.ai:

ubuntu@ip-172-31-59-112:~$ curl --version
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

buildkg2.rtx.ai:

ubuntu@ip-172-31-62-73:~/RTX-KG2$ curl --version
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd
ecwood commented 1 year ago

This completed successfully in the KG2.8.4pre build so I am going to close out this issue.