Add `metaharvester` - Githubissues

TCP-Lab / x.FASTQ

Bash wrapper modules for the remote analysis of RNA-Seq data, with persistency features.

MIT License

2 stars 0 forks source link

Add `metaharvester` #8

Closed MrHedmad closed 8 months ago

MrHedmad commented 9 months ago

This PR adds the new script metaharvest.

This script does a few things related to metadata:

It can download the file needed by getFASTQ for a whole ENA dataset by just specifing the ID of the dataset;
It can download and parse the GEO metadata (if any) of an ENA dataset and merge it in with the ENA metadata to create a whole metadata file to ease downstream processing.

More info is in metaharverst -h.

Issues to fix before merging

[x] There is no logic to add the ENA sample IDs to the GEO metadata;
[x] Documentation needs to be edited;
[x] x.FASTQ needs to be aware of metaharvest for, e.g., symlinking;

Do not merge this PR until the above are fixed! (I cannot make this a draft PR since the repository is set to private).

MrHedmad commented 9 months ago

The x.FASTQ -r command does not seem to pickup metaharvest, but x.FASTQ -l does work. I'm not sure why.

Feat-FeAR commented 9 months ago

Thanks for the addon! I've just started looking at it...

Feat-FeAR commented 9 months ago

The x.FASTQ -r command does not seem to pickup metaharvest, but x.FASTQ -l does work. I'm not sure why.

This is simply because the for loop in x.fastq.sh that collects the versions of all the modules breaks down before it gets to the harvester... and this is due to the fact that it is an implicit requirement of ALL x.FASTQ scripts to have somewhere a versioning line of the type

ver="x.y.z"

It may well be a comment, but it has to be there. See here, for instance: https://github.com/TCP-Lab/x.FASTQ/blob/299b544fb94a8596ebc4652d098fbfecc5f5b44d/cc_assembler.R#L17-L19

MrHedmad commented 8 months ago

The x.FASTQ -r command does not seem to pickup metaharvest, but x.FASTQ -l does work. I'm not sure why.

This is simply because the for loop in x.fastq.sh that collects the versions of all the modules breaks down before it gets to the harvester... and this is due to the fact that it is an implicit requirement of ALL x.FASTQ scripts to have somewhere a versioning line of the type
ver="x.y.z"
It may well be a comment, but it has to be there. See here, for instance:

https://github.com/TCP-Lab/x.FASTQ/blob/299b544fb94a8596ebc4652d098fbfecc5f5b44d/cc_assembler.R#L17-L19

I've added versions for all the scripts, so this is fixed, but it might be nice to not have the script fail if they are not present, but this is for another PR.