biomaj2gal end up with bioblend connection error

[nagoue]

Hi @abretaud, Hope you're doing well!

Not sure if I post the issue at the right place ... let me know, if I should post somewhere else like data_manager_manual or bioblend github ? We are using Galaxy 19.09, a postgreSQL DB and the last biomaj2galaxy release.

When running bioMAJ with properties file containing biomaj2galaxy, I got the following log error :

...
PROCESS:ERROR:GAL_dm_galaxy_dm
INFO  [root][Thread-37] PROCESS:EXEC:GAL_dm_galaxy_dm:False
DEBUG [root][Thread-37] Process:progress:GAL_dm_galaxy_dm=False
DEBUG [root][MainThread] Meta threads are over
ERROR [root][MainThread] Error during task postprocess
...

GAL_dm_galaxy_dm.err file end up with :

File \\"lib/galaxy/web/framework/decorators.py\\", line 282, in decorator\\n    
rval = func(self, trans, *args, **kwargs)\\n  
File \\"lib/galaxy/webapps/galaxy/api/jobs.py\\", line 135, in show\\n    
job = self.__get_job(trans, id)\\n  
File \\"lib/galaxy/webapps/galaxy/api/jobs.py\\", line 135, in show\\n    
job = self.__get_job(trans, id)\\n  
File \\"lib/galaxy/webapps/galaxy/api/jobs.py\\", line 475, in __get_job\\n    
return self.job_manager.get_accessible_job(trans, decoded_job_id)\\n  
File \\"lib/galaxy/managers/jobs.py\\", line 56, in get_accessible_job\\n    
raise ObjectNotFound()\\nObjectNotFound: No such object found.\\n" 
"err_msg": "No such object found.", "err_code": 404001}', 0 attempts left: {"traceback": "Traceback (most recent call last):\n  
File \"lib/galaxy/web/framework/decorators.py\", line 282, in decorator\n    
rval = func(self, trans, *args, **kwargs)\n  
File \"lib/galaxy/webapps/galaxy/api/jobs.py\", line 135, in show\n    
job = self.__get_job(trans, id)\n  
File \"lib/galaxy/webapps/galaxy/api/jobs.py\", line 475, in __get_job\n    
return self.job_manager.get_accessible_job(trans, decoded_job_id)\n 
File \"lib/galaxy/managers/jobs.py\", line 56, in get_accessible_job\n    
raise ObjectNotFound()\nObjectNotFound: No such object found.\n" 
"err_msg": "No such object found.", "err_code": 404001}

Part of my properties file look like this :

...
#Post-process configuration for BIOMAJ2GALAXY using data manager and data libraries
GALAXY.db.post.process=GAL_dm,GAL_dl_gff3,GAL_dl_gtf

GAL_dm=galaxy_dm

galaxy_dm.name=galaxy_dm
galaxy_dm.desc=Add files to Galaxy tool data tables
galaxy_dm.type=galaxy
galaxy_dm.exe=biomaj2galaxy
galaxy_dm.args=-v -f /storage/galaxyprod/.bm2g.yml add -d "${localrelease}" -n "ensembl triticum aestivum (${remoterelease})" --no-file-check -g fasta/all.fasta bowtie2:bowtie2/all twobit:2bit/all.2bit bwa:bwa/all bwa_mem:bwa/all star:star/all "hisat2:hisat2/all/${db.name}" "blastdb:blast/dna/${db.name}_dna:ensembl triticum aestivum dna" "blastdb:blast/cdna/${db.name}_cdna:ensembl triticum aestivum cdna" "blastdb_p:blast/pep/${db.name}_pep:ensembl triticum  aestivum pep" 
galaxy_dm.cluster=false
...

We have tried restarting Galaxy and Postgres services, modifications on Galaxy server in config/galaxy.yml like threads: 8 -> 1 offload-threads:8 -> 1

but that didn't solve the problem.

Any suggestions are very welcome.

All the best,

[abretaud]

¡Hola Nadia! That's... strange... Which version of bioblend are you using? (pip freeze) Is it a 19.09 with all the bug fixes from the https://github.com/galaxyproject/galaxy/tree/release_19.09 branch? Did it ever work at some point with the same versions?

[nagoue]

bioblend==0.13.0 We did the upgrade to 19.09 this end of June. Should I look for special bug fixes ? It hasn't work well since the update :(

[abretaud]

No specific bugfix no, just that there were probably some fixes on this branch after the official release, so... it's best to be up-to-date Would updating to 20.01 be a problem?

[nagoue]

We usually prefer to stay one year behind the current release. We already updated from 19.05 to 19.09 to fix compatibility bug with biomaj ... It will take time before we can schedule an update to 20.01 from now and we are not sure that will solve the problem, right ?

[abretaud]

Yes, not sure if a 20.01 update would solve it. The error is quite strange, I never had something like this... Do you see anything special in the logs? Is there some data_manager job being run or not? Can you test if you manage to use bioblend to submit/track normal jobs?

[nagoue]

Hi @abretaud, Solving #6 helped me to go a step further. I posted an issue #3302 on galaxyproject/tools-iuc/ that maybe connected to the job error I now get. So I guess this is OK to close this issue ?

[abretaud]

Ok, let's close this one! Sorry for not being very responsive on that, it's hard to find the time to debug the current issue :/