geneontology / go-graphstore

Gene Ontology RDF GraphStore
3 stars 4 forks source link

Provision #19

Closed abessiari closed 3 years ago

abessiari commented 3 years ago

Added docker file and aws provisionning.

See provision/PROVISION_README.md for local testing and PROVISION_AWS_README.md for aws.

kltm commented 3 years ago

@abessiari I've started testing this and it's looking pretty good. One thing I noticed, when working with a larger query, was that I seemed to get timeouts after a few (5?) seconds: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryInterruptedException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.bigdata.bop.engine.QueryTimeoutException: Query deadline is expired. We'll probably want to parameterize timeouts. I was looking to see how to do that, but couldn't figure out where the config files were (https://github.com/geneontology/go-graphstore/pull/19/files#r749868678).

abessiari commented 3 years ago

@kltm

conf/readonly_cors.xml there is timeout set to 30 seconds.

queryTimeout 30000
kltm commented 3 years ago

@abessiari Ah, thank you--I'm not sure how I missed that.

kltm commented 3 years ago

@abessiari With the testing that I'm doing and the timeouts I keep hitting, I think it would be useful if we could parameterize the queryTimeout like we do for the RAM. If that's tricky, I'll try and make up an SOP so we can do this at need.

kltm commented 3 years ago

@abessiari Similar to the reported issue with the other provisioning, I'm running into:

sjcarbon@moiraine:~/local/src/git/go-graphstore-abessiari/provision[provision]$:( ansible-playbook -e "stage_dir=$STAGE_DIR" -u ubuntu -i "$HOST," --private-key $PRIVATE_KEY build_image.yaml 

PLAY [build docker image] ******************************************************

TASK [Gathering Facts] *********************************************************
fatal: [52.5.59.113]: FAILED! => {"changed": false, "module_stderr": "/home/sjcarbon/.ssh/config line 3: Unsupported option \"rsaauthentication\"\r\nShared connection to 52.5.59.113 closed.\r\n", "module_stdout": "/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "rc": 127}

fixed with

ssh -i KEY_PATH login@location 'sudo apt-get update && sudo apt-get -y install python'
kltm commented 3 years ago

Similarly, as with AmiGO, replace the unpacking module with a manual tar pigz command and look to use ansible (or something, even automatic) to start the docker images up.

kltm commented 3 years ago

I can confirm that the "dual" CNAME feature seems to work as advertised so far!

kltm commented 3 years ago

Completed testing cycle: looking good! Is it better to merge now before more work goes on here, or do you want to keep this PR open?

abessiari commented 3 years ago

I am working on the timeout .... Let us wait.

abessiari commented 3 years ago

@kltm

done. we can discuss on the meeting.

https://github.com/geneontology/go-graphstore/pull/19/commits/1323c82e6eb8823a628192f323223772d1d1bfc5

kltm commented 3 years ago

Under discussion, merging now.