search

claraqin / neonMicrobe

Processing NEON soil microbe marker gene sequence data into ASV tables.
GNU Lesser General Public License v3.0
9 stars 4 forks source link

New server setup data download #2

Closed yyachung closed 4 years ago

yyachung commented 4 years ago

I'm just starting to go through the new server setup instructions, and realized that it pertains specifically to downloading data from the Zhu lab space on the UCSB socs-stats server. Do you want us to directly test that (in which case I'll somehow need access to the server) or can the utils.R code be generalized for direct download from NEON via neonUtilities? Thanks!

KaiZhuPhD commented 4 years ago

Good point. The idea is not to depend on our server, but for everyone to download to a local machine (Windows or Mac). @claraqin is working on it.

claraqin commented 4 years ago

Hi Anny, thanks for pointing this out. I've been meaning to address it. For now, you can get around most of these issues by replacing /data/ZHULAB/NEON_DOB/ with your own directory wherever it appears in the Wiki and in the parameters at the top of utils.R. I'll make this clarification in the Wiki and let you know when I do; regardless, replacing the /data/ZHULAB/NEON_DOB/ with your directory is something that you have to do.

yyachung commented 4 years ago

Somehow I did not read through the whole thing and realize that it's the out directory that's being specified. Thanks for the tip, currently downloading :) Edit: downloaded fine but failed to unpack and stack

claraqin commented 4 years ago

Hi Anny, thanks for following up! I realized that this is because there is nothing in the script to guarantee that the existing directory structure is consistent with the user-specified one, which causes issues when downloading raw data. I've fixed this by adding a few lines to the beginning of utils.R:

# If preset output directories do not exist, create them
if(!dir.exists(preset_outdir_sequence)) dir.create(preset_outdir_sequence, recursive=TRUE)
if(!dir.exists(preset_outdir_seqmeta)) dir.create(preset_outdir_seqmeta, recursive=TRUE)
if(!dir.exists(preset_outdir_soil)) dir.create(preset_outdir_soil, recursive=TRUE)

This should take care of the issue. If not, please re-open this thread.

Also, I've edited the "New server setup" Wiki page so it should hopefully be more intuitive to a non-UCSC user now.