Add scripts, tests, envs, docs for the preparation of pipeline execution

AngryMaciek commented 4 years ago

Description

Major update from Maciek [1/2]: added files related to the preparation of pipeline execution (scripts prior snakemake run).

Type of change

[x] New feature (non-breaking change which adds functionality)
[x] Documentation updated

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have not reduced the existing code coverage

AngryMaciek commented 4 years ago

It seems to me that we have a problem with the Travis CI...

Whenever I execute the CI test commands on my local machine everything is fine, I have never experienced any problems. On the Travis servers we get an error while trying to download and unzip a database within a bash script. The download is with curl, the extraction with unzip commands. I was trying to inspect the issue a little so I have downloaded that database on my local machine, added it to the repository under resources directory and added a respective CI test to unzip the already-downloaded version. And this is very strange - please take a look at the CI Details in this Pull Reqest (build 35) - it appears as if the extraction of the already-downloaded version went fine whereas the extraction after downloading on Travis machine raised an error. This is confusing since the unzipping commands are the same...

@krish8484 , could you please take a look? Is something wrong with the curl download?

EDIT: While you'll be inspecting the YAML file you will notice how I prepare the environment fo the pipeline execution. I guess it might be a good time to intruduce you to the conda package manager (if you do not know it already). Please take a look at these short videos:
https://www.youtube.com/watch?v=clKgaEZJYww https://www.youtube.com/watch?v=ZXrUfNwPOe4 What we essentially do in the CI is download and install the miniconda package manager, build and activate a dedicated virtual environment for our pipeline. I was following these notes with some modifications:
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/use-conda-with-travis-ci.html

krish8484 commented 4 years ago

I searched around a bit, and I also tried implementing it. The unzip is not the problem, the zip file is not getting downloaded in the first place. I tried using wget instead of curl, I changed the script but it returns a 403 request. Link to the build. The breaking command works in my local machine, this must be something related to travis maybe.

AngryMaciek commented 4 years ago

Thanks. In that case I will comment out this test line with the bash script. I have also checked it on mac and linux and it works, its rather plausible then that this issue is sth. related to Travis machine.

zavolanlab / bindz-rbp