Parallelization of run-data-managers. Add galaxy-wait to tests. Fix galaxy-wait issue with nginx.

galaxyproject / ephemeris

Library for managing Galaxy plugins - tools, index data, and workflows.

https://ephemeris.readthedocs.org/

Other

27 stars 38 forks source link

Parallelization of run-data-managers. Add galaxy-wait to tests. Fix galaxy-wait issue with nginx. #55

Closed rhpvorderman closed 6 years ago

rhpvorderman commented 6 years ago

fixes #54

The stdout of run-data-managers is not up to scratch though. It does not give any output. If the -v flag is used, it gives too much output (also debug messages). Can somebody show me how to get run-data-managers too display all the info messages in stdout?

rhpvorderman commented 6 years ago

Found the line to enable more output.

rhpvorderman commented 6 years ago

Moved the "data manager still running" to debug as it clutters output. Removing items from a list while iterating over a list was an ugly mistake I made which caused skipping over every other job. (I did not notice it earlier since the tests ran ok.) This is now corrected. Everything should be in order now.

rhpvorderman commented 6 years ago

Test failed because of random error in the docker container. Updated documentation to reflect changes made to the installer and to trigger new testing.

rhpvorderman commented 6 years ago

Because I was impatient I already used this change in production and it works. We used it to run 10 genomes on 7 data managers. It was quite satisfying to see these jobs pushed to galaxy ten at a time instead of one by one.

Can somebody review this? Thanks in advance!

rhpvorderman commented 6 years ago

I added some more verbose information and more error checking. Testing on travis seems to yield inconsistent results. It does not seem to be related to the run-data-managers function. Could somebody help me with this @mvdbeek perhaps?

rhpvorderman commented 6 years ago

Did several things:

Tie in galaxy-wait in the testing. This also removes some sleep commands which should speed up testing and abolish unnecessary waiting.
fix errors in galaxy-wait (sleep.py) that resulted from the docker container returning a nginx forbidden error.
Update bioblend. In #53 the tests were changed because of an error in bioblend. This was fixed on https://github.com/galaxyproject/bioblend/pull/251. With the new version tests can be run both on api key and user/password again on python3.
added a small wait step to give the data tables a chance to reload between two run-data-manager commands. This should happen near-instant, but since the two run-data-managers commands were also run near-instantly after each other this seemed not to work on travis. A very conservative wait of 15s was added in between.

rhpvorderman commented 6 years ago

These random errors that are due to the galaxy in docker not updating fast enough in travis are somewhat frustrating.

rhpvorderman commented 6 years ago

Retry attempts did not result in success. The cause of the error is that while the viruses are downloaded and put in the data table, some virus do not show up in the data table. I think this is caused by the viruses being added simultaneously to the data table by different handlers. This might cause the attempt not to be registered. I tried to run the test container with only handler process to check if this fixes the problem.

rhpvorderman commented 6 years ago

An error occurred while generating the build script.

bgruening commented 6 years ago

I think travis has some problems, I see this elsewhere as well :(

bgruening commented 6 years ago

I will restart the test a few times in the next hours and see whats going on. There is also a new Galaxy Docker release, which might help or break things ... let's see. Eitherway - thanks a lot for this work, looks great!

bgruening commented 6 years ago

1/6 or so failed. This is still annoying but it s not that bad. Maybe its better with the new release?

Eitherway, thanks a lot!