Error in Nginx because of missing parameter

[Prakash2403]

Nginx requires MANAGER_HOST parameter, which is missing from config file.

I tried adding MANAGER_HOST=127.0.0.1, but then it raised another issue 2019/11/29 10:02:21 [emerg] 1#1: host not found in upstream "interfaces:6010" in /etc/nginx/conf.d/default.conf:8

How to fix this issue?

[YaphetKG]

1. using MANAGER_HOST=127.0.0.1 is that since you are running nginx as a container that would refer to the container it self (localhost), so it would not point to the real manager container. So use the container name.
2. Interfaces is another service which the manager depends on and it can be found and deployed from here github.com/NCATS-Gamma/robokop-interfaces. Once you have that service the host names for manager, interfaces, ranker all declared in the env file for robokop would be their corresponding container names if you want to run all of them on a single host.
   Please let us know if that works.

[Prakash2403]

@YaphetKG

I have cloned all three repos: robokop, robokop-interfaces and robokop-rank. I want to run all of them on single machine. I am facing two major issues:

1. Manager services are unable to connect to rabbitMQ services. I can't figure out the reason.
2. ModuleNotFoundError: No module named 'rdkit' error in interfaces server and writer.

My robokop.env file contents:

#################### ROBOKOP Environmental Variables ####################

# Web Interface and API
ROBOKOP_HOST=127.0.0.1
ROBOKOP_PROTOCOL=http
MANAGER_PORT=80
MANAGER_HOST=manager
# PUBMED

PUBMED_CACHE_PORT=9005

# Number of celery works to answer tasks and rebuild graphs
MANAGER_NUM_ANSWER_WORKERS=4
MANAGER_NUM_BUILDER_WORKERS=4

# Flask Security settings for user creation, mail etc.
#ROBOKOP_SECRET_KEY
#ROBOKOP_SECURITY_PASSWORD_SALT
#ROBOKOP_MAIL_SERVER
#ROBOKOP_MAIL_USERNAME
#ROBOKOP_MAIL_PASSWORD
#ROBOKOP_DEFAULT_MAIL_SENDER
#ROBOKOP_SECURITY_EMAIL_SENDER

# Web Interface Storage - Users, Questions and Answers - Postgres
POSTGRES_HOST=postgres
# Location here is referenced by container name on the local docker network
POSTGRES_PORT=5454
POSTGRES_USER=murphy
POSTGRES_DB=robokop
#POSTGRES_PASSWORD
#ADMIN_EMAIL
#ADMIN_PASSWORD

GRAPHQL_HOST=graphql
GRAPHQL_PORT=6423

# NPL Server - Future
NLP_HOST=nlp
NLP_PORT=9475

# Graph DB - NEO4j
NEO4J_HOST=neo4j
# This references an external instance
# The machine hosting this env file will not start the graph container
NEO4J_HTTP_PORT=7474
NEO4J_HTTPS_PORT=7473
NEO4J_BOLT_PORT=7687
NEO4J_HEAP_MEMORY=32G
# Only used by the machine that runs this container
NEO4J_HEAP_MEMORY_INIT=32G
# Only used by the machine that runs this container
NEO4J_CACHE_MEMORY=32G
# Only used by the machine that runs this container
#NEO4J_PASSWORD

# HTTP Cache - Redis
CACHE_HOST=request_cache
# Externally hosted request cache
CACHE_PORT=6380
CACHE_DB=0
#CACHE_PASSWORD

# Builder API - robokop-interfaces - Flask server with celery workers
BUILDER_HOST=interfaces
BUILDER_PORT=6010
BUILDER_NUM_WORKERS=4

# Builder Internal Communication Cache - Redis
BUILD_CACHE_HOST=build_cache
BUILD_CACHE_PORT=6379
BUILD_CACHE_DB=1
# Port is not publish outside of containers, no password is applied

# Celery Queue Results Backend - Redis
RESULTS_HOST=celery_results
RESULTS_PORT=6381
#RESULTS_PASSWORD
MANAGER_RESULTS_DB=0
BUILDER_RESULTS_DB=1
RANKER_RESULTS_DB=2

# Celery Message Broker - RabbitMQ
BROKER_HOST=broker
BROKER_PORT=5672
BROKER_MONITOR_PORT=15672
BROKER_USER=murphy
#BROKER_PASSWORD

# Ranking API - robokop-rank - Flask server, Open API and with celery workers
RANKER_HOST=rank
RANKER_PORT=6011
RANKER_NUM_WORKERS=4

# Literary Co-occurence Server for ranking
OMNICORP_HOST=omnicorp
#omnicorp
OMNICORP_PORT=5433
OMNICORP_DB=robokop
OMNICORP_USER=murphy
#OMNICORP_PASSWORD

# Supervisor
# Manager, Builder and Ranker all user supervisor to manager celery daemon
MANAGER_SUPERVISOR_PORT=9001
BUILDER_SUPERVISOR_PORT=9002
RANKER_SUPERVISOR_PORT=9003
SUPERVISOR_USER=admin
#SUPERVISOR_PASSWORD=""

COMPOSE_PROJECT_NAME=robokop

#############################################################
#############################################################
ADMIN_EMAIL=prakash@aganitha.ai
ADMIN_PASSWORD="hello-world"
POSTGRES_PASSWORD="hello-world"
NEO4J_PASSWORD="hello-world"
CACHE_PASSWORD="hello-world"
RESULTS_PASSWORD="hello-world"
BROKER_PASSWORD="hello-world"
SUPERVISOR_PASSWORD="hello-world"
OMNICORP_PASSWORD="hello-world"
PUBMED_CACHE_PASSWORD="hello-world"
ROBOKOP_SECRET_KEY="hello-world"
ROBOKOP_SECURITY_PASSWORD_SALT="hello-world"
ROBOKOP_MAIL_SERVER="hello-world"
ROBOKOP_MAIL_USERNAME="hello-world"
ROBOKOP_MAIL_PASSWORD="hello-world"
ROBOKOP_DEFAULT_MAIL_SENDER="hello-world"
ROBOKOP_SECURITY_EMAIL_SENDER="hello-world"

How to fix this issue?

Thanks for the help :)

[Prakash2403]

Also, there are two neo4j containers, one in robokop-root/robokop-interfaces/deploy/graph/docker-compose.yml and another in robokop-interfaces/deploy/robokopkg/docker-compose.yml. Both of them have container name as neo4j, hence there is a conflict. One obvious way to avoid the conflict is to have different names, but both of them connect to same port (7687) on machine.

How to resolve this?

[YaphetKG]

Hi @Prakash2403 About the neo4j instances: you'd not need both of those Neo4j instances running, the instance under robokopkg is used for hosting read-only instance. But if you'd like to write to it please use the one under graph. For your first two questions:

1. Are the backend services (located at robokop\robokop\deploy\backend\docker-compose services running ?
2. There was a new push to robokop-interfaces@master branch. We made a new conda based docker file. So running a docker-build once again should refresh the image and fix the issue.

[Prakash2403]

Hi @YaphetKG , got it. Thanks for the help.

I was running the script for Initial Database setup, as described in robokop-interfaces repo. It ran fine for a some time, then raised the following error:

Failed to write data to connection Address(host='neo4j', port=7687) (Address(host='172.18.0.11', port=7687)); ("0; 'Underlying socket connection gone (_ssl.c:2283)'")

Is this because of memory issues? Relevant neo4j params are:

NEO4J_HEAP_MEMORY=1G
# Only used by the machine that runs this container
NEO4J_HEAP_MEMORY_INIT=1G
# Only used by the machine that runs this container
NEO4J_CACHE_MEMORY=1G

The complete log is:

adding node PO to concept gross_anatomical_structure
   adding node KEGG.PATHWAY to concept pathway
   adding node EFO to concept genetic_condition
   adding node GO to concept biological_process_or_molecular_activity
   adding node PHARMGKB.PATHWAYS to concept pathway
   adding node CHEBI to concept metabolite
   adding node DOID to concept genetic_condition
   adding node UNII to concept metabolite
   adding node UBERON to concept gross_anatomical_structure
   adding node NCBIGENE to concept gene
   adding node REACTOME to concept pathway
   adding node WIKIPATHWAYS to concept pathway
   adding node CL to concept cell
   adding node BIOCARTA.PATHWAY to concept pathway
   adding node HGNC.FAMILY to concept gene_family
   adding node HP to concept phenotypic_feature
   adding node PR to concept macromolecular_complex
   adding node INCHI to concept metabolite
   adding node ENSEMBL to concept noncoding_RNA_product
   adding node ORPHANET to concept genetic_condition
   adding node NCIT to concept phenotypic_feature
   adding node DRUGBANK to concept metabolite
   adding node CLO to concept cell_line
   adding node SMPDB to concept pathway
   adding node HMDB to concept metabolite
   adding node FAO to concept gross_anatomical_structure
   adding node HGNC.SYMBOL to concept gene
   adding node INCHIKEY to concept metabolite
   adding node KEGG.DRUG to concept metabolite
   adding node OMIM to concept genetic_condition
   adding node DBSNP to concept sequence_variant
   adding node ZFIN to concept gene
   adding node KEGG.COMPOUND to concept metabolite
   adding node PUBCHEM.COMPOUND to concept metabolite
   adding node PANTHER.FAMILY to concept gene_family
   adding node HGNC to concept gene
  -+ anatomical_entity anatomical_entity link: anatomy_to_anatomy_type op: uberongraph.get_anatomy_by_anatomy_graph
  -+ anatomical_entity gene link: anatomy_to_gene op: hetio.anatomy_to_gene
  -+ anatomical_entity phenotypic_feature link: anatomy_to_phenotype op: uberongraph.get_phenotype_by_anatomy_graph
  -+ anatomical_entity biological_process link: bio_process_anatomical_entity op: uberongraph.get_process_by_anatomy
  -+ anatomical_entity disease link: anatomy_to_disease op: uberongraph.get_disease_by_anatomy_graph
  -+ anatomical_entity molecular_activity link: anatomy_to_biological_process_or_activity op: uberongraph.get_activity_by_anatomy
  -+ biological_process_or_activity gene link: bio_process_gene op: quickgo.go_term_to_gene_annotation
  -+ biological_process_or_activity anatomical_entity link: bio_process_anatomy op: uberongraph.get_anatomy_by_process_or_activity
  -+ biological_process_or_activity chemical_substance link: bio_process_chemical_substance op: uberongraph.get_chemical_entity_by_process_or_activity
  -+ chemical_substance gene link: chemical_targets_gene op: chembio.graph_pubchem_to_ncbigene
  -+ chemical_substance gene link: drug_to_gene op: ctd.drug_to_gene_expanded
  -+ chemical_substance gene link: drug_to_enzyme op: mychem.get_gene_by_drug
  -+ chemical_substance gene link: chem_to_precursor op: gtopdb.chem_to_precursor
  -+ chemical_substance gene link: ligand_to_gene op: gtopdb.ligand_to_gene
  -+ chemical_substance disease_or_phenotypic_feature link: adverse_events op: mychem.get_adverse_events
  -+ chemical_substance disease_or_phenotypic_feature link: drugcentral op: mychem.get_drugcentral
  -+ chemical_substance chemical_substance link: chemical_to_chemical_class op: uberongraph.get_chemical_by_chemical
  -+ chemical_substance chemical_substance link: complex_to_part op: gtopdb.complex_to_part
  -+ chemical_substance food link: chemical_substance_to_food op: foodb.food_to_chemical_substance
  -+ disease_or_phenotypic_feature chemical_substance link: exposures op: ctd.disease_to_exposure
  -+ disease_or_phenotypic_feature chemical_substance link: disease_to_chemical op: ctd.disease_to_chemical
  -+ disease chemical_substance link: uber_disease_to_chemical op: uberongraph.get_chemical_by_disease
  -+ disease phenotypic_feature link: disease_to_phenotype op: biolink.disease_get_phenotype
  -+ disease phenotypic_feature link: disease_to_phenotype op: hetio.disease_to_phenotype
  -+ disease gene link: disease_to_gene op: pharos.disease_get_gene
  -+ disease gene link: disease_to_gene op: biolink.disease_get_gene
  -+ disease metabolite link: disease_to_metabolite op: hmdb.disease_to_metabolite
  -+ disease biological_process_or_activity link: disease_to_biological_process_or_activity op: uberongraph.get_process_by_disease
  -+ disease anatomical_entity link: disease_to_anatomical_entity op: uberongraph.get_anatomy_by_disease
  -+ drug gene link: drug_to_gene op: pharos.drug_get_gene
  -+ metabolite gene link: metabolite_to_gene op: hmdb.metabolite_to_enzyme
  -+ metabolite gene link: chemical_to_gene op: kegg.chemical_get_enzyme
  -+ metabolite disease link: metabolite_to_disease op: hmdb.metabolite_to_disease
  -+ metabolite pathway link: metabolite_to_pathway op: hmdb.metabolite_to_pathway
  -+ metabolite metabolite link: metabolite_to_metabolite op: kegg.chemical_get_chemical
  -+ gene anatomical_entity link: gene_to_anatomy op: hetio.gene_to_anatomy
  -+ gene disease link: gene_to_disease op: biolink.gene_get_disease
  -+ gene disease link: gene_to_disease op: hetio.gene_to_disease
  -+ gene disease link: gene_to_disease op: pharos.gene_get_disease
  -+ gene gene_family link: gene_to_gene_family op: hgnc.gene_get_gene_family
  -+ gene cellular_component link: gene_to_cellular_component op: quickgo.gene_to_cellular_component
  -+ gene chemical_substance link: gene_to_drug op: ctd.gene_to_drug_expanded
  -+ gene chemical_substance link: gene_get_drug op: pharos.gene_get_drug
  -+ gene metabolite link: enzyme_to_metabolite op: hmdb.enzyme_to_metabolite
  -+ gene pathway link: gene_to_pathway op: biolink.gene_get_pathways
  -+ gene pathway link: gene_to_pathway op: chembio.graph_get_pathways_by_gene
  -+ gene biological_process_or_activity link: biological_process_or_activity op: biolink.gene_get_process_or_function
  -+ gene phenotypic_feature link: gene_get_phenotype op: biolink.gene_get_phenotype
  -+ pathway gene link: pathway_to_gene op: biolink.pathway_get_gene
  -+ pathway metabolite link: pathway_to_metabolite op: hmdb.pathway_to_metabolite
  -+ phenotypic_feature anatomical_entity link: phenotype_to_anatomy op: uberongraph.get_anatomy_by_phenotype_graph
  -+ phenotypic_feature disease link: phenotype_to_disease op: biolink.phenotype_get_disease
  -+ phenotypic_feature biological_process link: phenotype_to_biological_process_or_activity op: uberongraph.get_process_by_phenotype
  -+ phenotypic_feature molecular_activity link: phenotype_to_biological_process_or_activity op: uberongraph.get_activity_by_phenotype
  -+ phenotypic_feature chemical_substance link: phenotype_to_chemical_substance op: uberongraph.get_chemical_by_phenotype
  -+ sequence_variant gene link: sequence_variant_to_gene op: myvariant.sequence_variant_to_gene
  -+ sequence_variant gene link: sequence_variant_to_gene op: ensembl.sequence_variant_to_gene
  -+ sequence_variant disease_or_phenotypic_feature link: sequence_variant_to_disease_or_phenotypic_feature op: gwascatalog.sequence_variant_to_disease_or_phenotypic_feature
  -+ sequence_variant sequence_variant link: sequence_variant_to_sequence_variant op: ensembl.sequence_variant_to_sequence_variant
  -+ gene_family gene link: gene_family_to_gene op: panther.get_gene_by_gene_family
  -+ gene_family biological_process_or_activity link: gene_family_to_biological_process_or_activity op: panther.get_biological_process_or_activity_by_gene_family
  -+ gene_family cellular_component link: gene_family_to_cellular_component op: panther.get_cellular_component_by_gene_family
  -+ gene_family gene_family link: gene_family_to_gene_family op: panther.get_gene_family_by_gene_family
  -+ gene_family pathway link: gene_family_to_pathway op: panther.get_pathway_by_gene_family
  -+ food chemical_substance link: food_to_chemical_substance op: foodb.food_to_chemical_substance
Failed to write data to connection Address(host='neo4j', port=7687) (Address(host='172.18.0.11', port=7687)); ("0; 'Underlying socket connection gone (_ssl.c:2283)'")
Failed to write data to connection Address(host='neo4j', port=7687) (Address(host='172.18.0.11', port=7687)); ("0; 'Underlying socket connection gone (_ssl.c:2283)'")

[YaphetKG]

I think that's an error that comes from the driver, I had opened an issue for it (https://github.com/neo4j/neo4j-python-driver/issues/293), but it doesn't really block writing to the db. I think you have a running instance now, check out the Graph MATCH (c:Concept) return * should give some results.

[Prakash2403]

Hi @YaphetKG

Thanks for all the help. I am able to run the tool on my system. :)

Just one last doubt, if I go to the following URL https://robokop.renci.org/questions/, then I can see a default question. To replicate that, I guess I have to import some neo4j db file. Can you please point me to the relevant file(s)?

Thanks for all the help :)

[YaphetKG]

You're welcome, You can find a couple of dumps over at robokopkg.renci.org. You can navigate through the pages and get the latest version being used by robokop.renci.org.

[Prakash2403]

Hi @YaphetKG , sorry to reopen the issue but found a couple of errors

1. Making a request to /api/node_properties raises a 500 Internal Server Error. Upon inspection, I found that a file named properties.json is missing. Complete log message:

   FileNotFoundError: [Errno 2] No such file or directory: '/home/murphy/robokop-interfaces/builder/api/../../greent/conf/properties.json'

2. Making a request to /api/search/disease also raises 500 Internal Server Error. Upon inspection, I found following log message:

neo4j.exceptions.ClientError: Failed to invoke procedure `db.index.fulltext.queryNodes`: Caused by: java.lang.IllegalArgumentException: There is no such fulltext schema index: node_name_index

How to deal with these?

I apologize for any inconvenience and thanks for all the help.

EDIT: I think second one may be resolved after importing some file, but what about 1st one?

[YaphetKG]

Thanks for point these out, will add these on the readme.

1. You'll need to run a post request to /api/node_properties and /api/predicates on robokop-manager
2. run docker exec -it interfaces python robokop-interfaces/scripts/setup_neo4j_index.py

[Prakash2403]

@YaphetKG

run docker exec -it interfaces python robokop-interfaces/scripts/setup_neo4j_index.py

It raises following error. Do I need to import the neo4j dump first?

Traceback (most recent call last):
  File "robokop-interfaces/scripts/setup_neo4j_index.py", line 13, in <module>
    predicates.remove('Unmapped_Relation')
ValueError: list.remove(x): x not in list

EDIT:

You'll need to run a post request to /api/node_properties and /api/predicates on robokop-manager

When I click on Ask a quick question on homepage, it makes a couple of GET requests to backend. Is there a need to change react code?

EDIT 2:

Commenting out predicates.remove('Unmapped_Relation') works. neo4j.exception.ClientException doesn't shows up now.

[YaphetKG]

Ok script should handle that case now, but I think it would be best if you run it after loading data.

[Prakash2403]

Okay, sure. Will try that also

[Prakash2403]

@YaphetKG

Ranker worker is failing because it requires MESSENGER_PORT and MESSENGER_HOST env variables, which are missing. What should be their value?

[Prakash2403]

@YaphetKG Any updates on MESSENGER HOST/PORT? Can't find anything related to that.

[YaphetKG]

Those should be declared in the environment files and their values should match the values assigned robokop-messenger/docker-compose.yml as a container name and the port (i.e if you are running it on the same host, else host will change to messenger's host and port should be 4868).

[Prakash2403]

@YaphetKG

Messenger is able to connect to omnicorp postgres DB, but the DB is empty. Perhaps, omnicorp DB initialization is missing?

I took a look at create_omnicorp.sql which seems to create relevant tables, but I am unable to find corresponding data files.

Here's a log sample

[2019-12-04 21:00:15,835: INFO/messenger.shared.omnicorp_postgres(MainProcess)]: Opening Connection to ROBOKOPDB Postgres
[2019-12-04 21:00:15,837: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(CHEBI:15367)...
[2019-12-04 21:00:15,838: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.chebi" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.CHEBI
                                    ^

Returning 0.
[2019-12-04 21:00:15,838: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(CHEBI:15939)...
[2019-12-04 21:00:15,839: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.chebi" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.CHEBI
                                    ^

Returning 0.
[2019-12-04 21:00:15,839: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(MONDO:0004992)...
[2019-12-04 21:00:15,839: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.mondo" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.MONDO
                                    ^

Returning 0.
[2019-12-04 21:00:15,839: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(MONDO:0005062)...
[2019-12-04 21:00:15,839: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.mondo" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.MONDO
                                    ^

Returning 0.
[2019-12-04 21:00:15,839: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(MONDO:0015262)...
[2019-12-04 21:00:15,840: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.mondo" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.MONDO
                                    ^

Returning 0.
[2019-12-04 21:00:15,840: DEBUG/messenger.modules.support(MainProcess)]: Computing OmnicorpSupport(MONDO:0000839)...
[2019-12-04 21:00:15,840: DEBUG/messenger.shared.omnicorp_postgres(MainProcess)]: OmniCorp query error: relation "omnicorp.mondo" does not exist
LINE 1: SELECT COUNT(pubmedid) from omnicorp.MONDO

[Prakash2403]

@YaphetKG

Any update(s) on data files? I found a couple of them in robokop-messenger, but not all. Ranker makes no sense without them.

Please help.

[Prakash2403]

@YaphetKG

Tried to import the latest dump: graph.9.9.19.dump.db.

Got this error. Please help.

robokop-interfaces/scripts/setup_neo4j_index.py:3: DeprecationWarning: The 'neo4j.v1' package is depr"""Get Neo4j index setup cypher."""
ecated, import from 'neo4j' instead
  from neo4j.v1 import GraphDatabase, basic_auth
Traceback (most recent call last):
  File "robokop-interfaces/scripts/setup_neo4j_index.py", line 11, in <module>
    predicates = list(row['predicate'] for row in result)
  File "robokop-interfaces/scripts/setup_neo4j_index.py", line 11, in <genexpr>
    predicates = list(row['predicate'] for row in result)
  File "/opt/conda/lib/python3.7/site-packages/neo4j/__init__.py", line 952, in records
    self._session.fetch()
  File "/opt/conda/lib/python3.7/site-packages/neo4j/__init__.py", line 527, in fetch
    detail_count, _ = self._connection.fetch()
  File "/opt/conda/lib/python3.7/site-packages/neobolt/direct.py", line 422, in fetch
    return self._fetch()
  File "/opt/conda/lib/python3.7/site-packages/neobolt/direct.py", line 464, in _fetch
    response.on_failure(summary_metadata or {})
  File "/opt/conda/lib/python3.7/site-packages/neobolt/direct.py", line 759, in on_failure
    raise CypherError.hydrate(**metadata)
neobolt.exceptions.DatabaseError: NOT PART OF CHAIN! RelationshipTraversalCursor[id=131071, open state with: denseNode=false, next=131071, mode=regular, underlying record=Relationship[131071,used=true,source=14701,target=2990,type=56,sPrev=131070,sNext=131069,tPrev=131860,tNext=130992,prop=611275,secondaryUnitId=-1,!sFirst,!tFirst]]