search

legumeinfo / ZZBrowse

Other
1 stars 1 forks source link

errors building GWAS datasets when installing de novo #36

Closed adf-ncgr closed 2 years ago

adf-ncgr commented 3 years ago

I'm trying to do the experiment of installing on a VM that doesn't have the obvious issues with freezing, but running into some issues on startup (this is with respect to the dev-legfedorg branch). First error is: Warning: Error in [: invalid subscript type 'list' 47: server [/srv/shiny-server/ZZBrowse/server.R#14] Error in dataFiles[sapply(dataFiles, function(f) !dir.exists(paste0(dataPath, : invalid subscript type 'list'

this seems to have to do with there being no ./www/config/data/ folder although if I mkdir it and restart the server with the folder extant it still gives the error. I can get around it by making a dummy folder within the directory, but then run into some issues seemingly related to changes in the datastore with marker file naming that will probably take a sit-down with Connor and DSCensor to resolve. However, if I copy the files from dev-legfedorg I can get around it for now (but onto other problems which I will save for other issues)

svengato commented 3 years ago

Yes, we need to bring DSCensor up to date with the other v2 data. I suspect mung bean is the only one that builds at the moment. Copying the www/config/data files is the best workaround for now.

adf-ncgr commented 3 years ago

that said, I think the first error is unrelated to issues of inconsistency between DSCensor and DS and probably needs a fix.

svengato commented 3 years ago

www/config/data is not in the repository because none of its contents are. There must be some way of adding an empty directory, I will look into it.

svengato commented 3 years ago

I mean only the directory.

adf-ncgr commented 3 years ago

OK, if that is not feasible maybe just have the code create it. Regarding mung bean, although that appeared to successfully download things from the datastore, the GWAS file that it constructed was empty other than the header. So I copied that one too, for now.

The possibly good news is that the resulting instance talking to gcv-microservices.lis.ncgr.org seems reasonably responsive after the initial server startup, at least so far. I think this is good news, but on the other hand it doesn't actually explain why the legfed VMs are so cranky about it.

svengato commented 2 years ago

I am getting around to this now, starting with the annotations.

  1. Updated the annotations URLs in the organism files. All of the URLs have changed to their v2 equivalent, but the mung bean and soybean files are the same as before (the others have changes).

  2. buildAnnotations.R used to have some special handling for the Name field in Medicago truncatula and mung bean. Now all except pigeonpea and soybean require the same handling as for mung bean: prepend "gensp." to the name.

This is implemented on dev-zzbrowse but not yet checked in, let me know if you see any problems.

svengato commented 2 years ago

(I know this has nothing to do with GWAS datasets, but it is another data store v2 issue.)

svengato commented 2 years ago

Fixed in commit 120c208 - we now use the v2 specifications. There are still some datasets in the data store that need fixing, see datastore-issues #64-68.