Replace cicar dataset in Conekt: Use Desi samples on Kabuli genome(cicar2)

[sdash-github]

A consequence of issue #8 Remove the LIS cicar1 dataset and use cicar2 instead. Cicar2 expression measures are based on mapping Desi(ICC) samples on Kabuli(CDC) genome. Refs: https://legumeinfo.org/lis_expression/all https://legumeinfo.org/feature/Cicer/arietinum_CDCFrontier/gene/cicar.CDCFrontier.gnm1.ann1.Ca_04638#pane=geneexpressionprofile

Keep in mind: Many tables will be affected.

[sdash-github]

At dev-conekt:

Copied mysql lis_conekt to lis_dev_conekt database. During mysqldump lost expr data and co-expr data of all species. Couldn't find a reason for this except that there wasn't enough memory in dev-conekt during the process.

Rectified gene tree file with Kabuli(CDCFrontier) gene names. Deleted cicar species(as Desi) and reloaded as Kabuli. Deleted cicar1 (Desi samples on Desi genome) expression data and reloaded cicar2 (Desi samples on CDCFrontier Kabuli genome). Deleted families. Reloaded. Deleted trees, reloaded rectified trees. Reconciled trees. dev-conekt now shows CDCFrontier data.

Reloaded expr data for other species (Co-expr data yet to be loaded for all the species).

Notes in '/projects/ConektCustomFamilyAndTree/note1-correctedCicerGeneNameKabuli.txt

[sdash-github]

At Conekt (Production) Reconcile trees proxy error

(https://conekt.lis.ncgr.org/)

Deleted cicar species(as Desi) via website (not db tables). (All network data and expression data lost during this process even for other species) Rreloaded species cicar as Kabuli. Reloaded expression data for cicar2 (Desi samples on CDCFrontier Kabuli genome); and phavu1, glyma1 and vigun1. Deleted families. Reloaded. "." vs. DOT acrobatics in gene families table. Deleted trees, reloaded rectified trees.

Reconciled trees.

https://conekt.lis.ncgr.org/admin/build/reconciled_trees/
ERROR:
"Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /admin_controls/reconcile/trees/.

Reason: Error reading from remote server"

FAILED: Reconcile trees.

[sdash-github]

Comments about Proxy Error

I also encountered this while deleting and adding gene families but db tables were emptied and filled anyway. During deleting families:

https://conekt.lis.ncgr.org/admin/families/
1   gene_family_members_from_database.UPDATED.forConektLoading.txt  18544
Deleted this via website.
ERROR:
"Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /admin/families/delete/.
Reason: Error reading from remote server"
!! But gene_families and sequence_family tables were emptied !!

During adding families:

ERROR:
"Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /admin_controls/add/family.

Reason: Error reading from remote server"
!! But gene_families table loaded !!

Also while adding gene descriptions:

ERROR:
"
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /admin_controls/add/sequence_descriptions.

Reason: Error reading from remote server
"
!! But gene descriptions were added and shows up in website. !!

[sdash-github]

Question for @adf-ncgr :\ Is there a way to see the server messages like Flask used to spill out to the screen where we had started the flash server? The /var/log/messages doesn't have anything from flash server.

[adf-ncgr]

I don't know offhand, but I'd suspect that there is a configuration option for logging, or maybe just enabling a debug mode that would give something equivalent. with respect to the proxy errors, I suspect it may be a case where something is timing out waiting for a response, but the task is completed anyway. Not sure I've completely understood the behavior you've reported here, though.

[sdash-github]

Trying to explain with clarity:

1. -- My Reconcile Trees process failed with the proxy error as noted above in comment titled, 'Conekt (Production) Reconcile trees proxy error' at https://github.com/legumeinfo/ConektLegumes/issues/11#issuecomment-578444777

-- In the next comment titled, 'Comments about Proxy Error' I was trying to make the point that other processes like deleting and adding families and adding gene description went through in spite of the proxy error.

I still haven't solved the Reconcile trees process.

2. About logging error messages a. In the config.py I have: 10 # Flask settings, make sure to turn DEBUG and TESTING to False for production 11 DEBUG = True 12 TESTING = True But I guess this is relevant for Flask server configuration. b. There is nothing about logging messages in conekt.wsgi c. In dev-conekt I had
   export FLASK_ENV=development for debugging in the webpage before flask run --host dev-conekt

I am not able to guess how to replicate this process in apache in the production environment.

[adf-ncgr]

OK, I had not understood that the reconciliation was actually failing to complete. I'm not certain that those config settings are only relevant for the flask development server, but it sounds like they aren't producing the desired result. If we can't figure out how to get it working under the apache setup, I guess the dev version is pointing to the same underlying database, so you could probably issue the same command through that alternative path and see if it reveals anything?

[sdash-github]

Now have two separate instances of the underlying database. I cloned the lis_conekt database into another instance called lis_dev_conekt so that I don't affect the production db during deletion of data and testing deletions via website. Reconcile tree process in dev_conekt did went through and I see expression in the tree context.

Found another location of error logs Per https://httpd.apache.org/docs/2.4/logs.html#errorlog , The error log is usually written to a file (typically error_log on unix systems I will look at this log file next time I attempt reconcile process in production.
Small disadvantage: The /var/log/httpd/error_log-xxxxxxx needs sudo.

May be we should think about sending the error logs to the /var/log/messages like in other servers we have. May be the ErrorLog directive at https://httpd.apache.org/docs/2.4/mod/core.html#errorlog can help in this. I am always apprehensive (rather scared) about changing these things for I think for understandable reasons-- not knowing enough I may not be able to handle if something goes wrong.

[adf-ncgr]

sorry, I thought you were already looking for things in the /var/log/httpd/error_log file; I think that is pretty standard for web error logging on our systems (on the ISU system there is a slightly differently named /var/log/httpd-error.log file). I think /var/log/messages is supposed to be for more generic system-level messages, although I think it is true that on the ISU servers they do get a bunch of web-related stuff written into them for some reason I don't entirely understand. I don't really understand the conventions in this regard very well either, I just have tried to adapt my habits to what seems to be the standard configuration behavior on the OSes that we are typically running in our VMs (and the VM hosts). We could chmod the log files so you can read them without sudo, at least on the dev machines (it is a pain, but presumably there is some security reason it is set that way, though probably nothing we need to be too worried about)

[sdash-github]

1. Couldn't debug production (conekt) wrt reconcile trees. Database=lis_conekt.
2. Pointed production to db in dev-conekt (= lis_dev_conekt) and now trees context works.

NEXT TODO: replace db lis_conekt with lis_dev_conekt

[sdash-github]

2021 Sep 24: RE-CHECK AFTER SITE FUNCTIONAL WHEN MYSQL DB REBUILT.