Banks don't load properly in biomaj-watcher after running --bank and --update

[markiskander]

Hey guys,

So currently I'm trying to add alu to biomaj-watcher. I took alu.properties and put it in the specified etc folder and then I ran biomaj-cli.py --config config path --bank alu --update

The output of that is: 2015-09-16 11:38:44,760 WARNI [root][MainThread] Bank:alu:Update 2015-09-16 11:38:44,760 INFO [root][MainThread] Bank:alu:Release:latest 2015-09-16 11:38:44,761 INFO [root][MainThread] Workflow:Start 2015-09-16 11:38:44,761 INFO [root][MainThread] Workflow:Start:init 2015-09-16 11:38:44,761 INFO [root][MainThread] Workflow:wf_init 2015-09-16 11:38:44,762 INFO [root][MainThread] Workflow:Start:check 2015-09-16 11:38:44,762 INFO [root][MainThread] Workflow:wf_check 2015-09-16 11:38:44,762 INFO [root][MainThread] Workflow:Start:depends 2015-09-16 11:38:44,762 INFO [root][MainThread] Workflow:wf_depends 2015-09-16 11:38:44,762 INFO [root][MainThread] Workflow:wf_depends:True 2015-09-16 11:38:44,763 INFO [root][MainThread] Workflow:Start:preprocess 2015-09-16 11:38:44,763 INFO [root][MainThread] Workflow:wf_preprocess 2015-09-16 11:38:44,763 INFO [root][MainThread] PROC:PRE 2015-09-16 11:38:44,766 INFO [root][MainThread] Workflow:Start:release 2015-09-16 11:38:44,766 INFO [root][MainThread] Workflow:wf_release 2015-09-16 11:38:44,766 INFO [root][MainThread] Workflow:wf_release:previous_session:None 2015-09-16 11:38:44,766 INFO [root][MainThread] Workflow:Start:download 2015-09-16 11:38:44,766 INFO [root][MainThread] Workflow:wf_download 2015-09-16 11:38:45,666 INFO [root][MainThread] Workflow:wf_download:release:remoterelease:2003-11-26 2015-09-16 11:38:45,666 INFO [root][MainThread] Workflow:wf_download:release:release:2003-11-26 2015-09-16 11:38:45,714 INFO [root][Thread-7] Start download thread 2015-09-16 11:38:45,715 INFO [root][Thread-8] Start download thread 2015-09-16 11:38:45,715 INFO [root][Thread-9] Start download thread 2015-09-16 11:38:45,717 INFO [root][Thread-10] Start download thread 2015-09-16 11:38:46,933 INFO [root][MainThread] Workflow:wf_download:Download:Threads:Over 2015-09-16 11:38:46,934 INFO [root][MainThread] Workflow:wf_uncompress 2015-09-16 11:38:46,976 INFO [root][MainThread] Workflow:wf_copy 2015-09-16 11:38:47,000 INFO [root][MainThread] Workflow:wf_copydepends 2015-09-16 11:38:47,001 INFO [root][MainThread] Workflow:Start:postprocess 2015-09-16 11:38:47,001 INFO [root][MainThread] Workflow:wf_postprocess 2015-09-16 11:38:47,002 INFO [root][MainThread] PROC:POST:BLOCK 2015-09-16 11:38:47,002 INFO [root][MainThread] PROC:POST:BLOCK:BLOCK1 2015-09-16 11:38:47,003 INFO [root][Thread-11] PROC:META:RUN:META0 2015-09-16 11:38:47,006 INFO [root][Thread-11] PROC:META:RUN:PROCESS:scanflatdir 2015-09-16 11:38:47,008 INFO [root][Thread-11] PROCESS:RUN:META0_scanflatdir 2015-09-16 11:38:47,183 INFO [root][Thread-11] PROCESS:EXEC:META0_scanflatdir:True 2015-09-16 11:38:47,184 INFO [root][Thread-11] PROC:META:RUN:PROCESS:savetogalaxy 2015-09-16 11:38:47,184 INFO [root][Thread-11] PROCESS:RUN:META0_savetogalaxy 2015-09-16 11:38:47,220 ERROR [root][Thread-11] PROCESS:ERROR:META0_savetogalaxy 2015-09-16 11:38:47,220 INFO [root][Thread-11] PROCESS:EXEC:META0_savetogalaxy:False 2015-09-16 11:38:47,223 ERROR [root][MainThread] Error during task postprocess 2015-09-16 11:38:47,223 INFO [root][MainThread] Workflow:wf_over 2015-09-16 11:38:47,224 INFO [root][MainThread] Notify:mark 2015-09-16 11:38:47,225 INFO [root][MainThread] BANK[alu] - STATUS[False] - UPDATE[True] - REMOVE[False] - RELEASE[2003-11-26] 2015-09-16 11:38:47,326 ERROR [root][MainThread] Could not send email: [Errno 111] Connection refused

And my alu.properties file is: [GENERAL] mail.user = 439618f42b537ee5c protocol = ftp mail.smtp.host = mailtrap.io historic.logfile.level = DEBUG scanflatdir.args = --scan $datadir/$dirversion/$localrelease --type=nucleic --tags="organism:human" bank.num.threads = 4 savetogalaxy.name = savetogalaxy elastic_index = biomaj savetogalaxy.native = data.stats = 1 savetogalaxy.exe = add_galaxy_library.py use_ldap = 0 block1.db.post.process = META0 savetogalaxy.desc = test remote.dir = /blast/db/FASTA/ dir.version = ncbi/blast/alu log.dir = /home/mark/custombiomaj/log meta0 = scanflatdir,savetogalaxy scanflatdir.types = mail.admin = mark savetogalaxy.format = files.num.threads = 4 offline.dir.name = offline/ncbi/blast/alu_tmp elasticnodes = localhost http.group.dir.name = 1 remote.files = ^alu..gz$ process.dir = /home/mark/custombiomaj/process use_elastic = 0 http.group.file.date = 2 data.dir = /home/mark/custombiomaj/banks ldap.host = localhost savetogalaxy.types = scanflatdir.docker = blocks = BLOCK1 scanflatdir.name = scanflatdir db.name = alu scanflatdir.desc = scan bank flat to detect available files scanflatdir.files = scanflatdir.format = http.group.dir.date = 2 http.parse.dir.line = <a[\s]+href=\"([\S]+)/\".alt=\"[DIR]\">.([\d]{2}-[\w\d]{2,5}-[\d]{4}\s[\d]{2}:[\d]{2}) http.parse.file.line = <a[\s]+href=\"([\S]+)\"._([\d]{2}-[\w\d]{2,5}-[\d]{4}\s[\d]{2}:[\d]{2})[\s]+([\d.]+[MKG]{0,1}) scanflatdir.native = savetogalaxy.files = celery.broker = mongodb://localhost:27017/biomaj_celery http.group.file.name = 1 scanflatdir.exe = scan.py scanflatdir.cluster = false db.url = mongodb://localhost:27017 visibility.default = public keep.old.version = 1 celery.queue = biomaj cache.dir = /home/mark/custombiomaj/cache savetogalaxy.cluster = false ldap.dn = nodomain db.type = nucleic_protein auto_publish = 0 server = ftp.ncbi.nih.gov http.group.file.size = 3 mail.password = 01f1e8b4712280 mail.from = biomaj@localhost db.fullname = alu.n : alu repeat element. alu.a : translation of alu.n repeats local.files = ^alu.(a|n).* ldap.port = 389 savetogalaxy.args = -u http://localhost:8080 -k c9bbb8c84e145e285f0b8b82f2cf6cd5 -l "$dbname" --lib-desc "just testing" -f "$dirversion" $datadir/$dirversion/$localrelease conf.dir = /home/mark/custombiomaj/etc savetogalaxy.docker =

So alu does show up in biomaj-watcher:

But when I go to edit, the fields are blank:

And when I edit in the fields manually and try to save, I get the following response:

And the console only gives the following message: 2015-09-16 11:54:10,013 WARNI [root][Dummy-1] db.formats is not set

so no error there.

Any ideas?

Thanks!

[osallou]

2 things:

• your postprocess save2galaxy failed during update

  2015-09-16 11:38:47,220 ERROR [root][Thread-11] PROCESS:ERROR:META0_savetogalaxy

you should be able to look at logs of the postprocess in log dir specified in config (check also that it is executable).

• Regarding properties edition in watcher:
  • alu.property needs to be readable/writable by the user running the biomaj-watcher
  • if you are not declared as "admin" in development.ini/production.ini, you must be the owner of the bank to view/modify its config (to avoid display of passwords etc...). Owner is set as the user running the update command via the command-line. It must match your login in the web interface. You can change the owner with the "--owner" option of the cmd line tool, but basically, you should not mix the updates by the bank, and the updates via the command-line, or always using the same user identifier.

=> either add your biomaj-watcher user as an admin in ini files => or create a different user matching your computer user identifier

[markiskander]

So first I made sure bioblend was installed. That was the reason save2galaxy failed. So the save2galaxy stopped.

Second I added my name to the list in development.ini (mark = admin) (this is the same name as the biomaj login and my computer login).

I modified alu manually and the changes appeared immediately in alu.properties, but again I get a 403 forbidden in the console. The output in the terminal is now:

## DEBUG protocol => ftp
## DEBUG BLOCK1.db.post.process => META0
## DEBUG scanflatdir.args => --scan $datadir/$dirversion/$localrelease --type=nucleic --tags="organism:human"
## DEBUG savetogalaxy.name => savetogalaxy
## DEBUG offline.dir.name => offline/ncbi/blast/alu_tmp
## DEBUG savetogalaxy.native => 
## DEBUG savetogalaxy.desc => test
## DEBUG remote.dir => /blast/db/FASTA/
## DEBUG dir.version => ncbi/blast/alu
## DEBUG scanflatdir.types => 
## DEBUG savetogalaxy.docker => 
## DEBUG savetogalaxy.exe => add_galaxy_library.py
## DEBUG META0 => savetogalaxy,scanflatdir
## DEBUG remote.files => ^alu.*\.gz$
## DEBUG savetogalaxy.types => 
## DEBUG scanflatdir.docker => 
## DEBUG blocks => BLOCK1
## DEBUG scanflatdir.name => scanflatdir
## DEBUG db.name => alu
## DEBUG scanflatdir.desc => scan bank flat to detect available files
## DEBUG scanflatdir.files => 
## DEBUG scanflatdir.format => 
## DEBUG scanflatdir.native => 
## DEBUG savetogalaxy.files => 
## DEBUG savetogalaxy.format => 
## DEBUG scanflatdir.exe => scan.py
## DEBUG scanflatdir.cluster => false
## DEBUG keep.old.version => 1
## DEBUG savetogalaxy.cluster => false
## DEBUG db.type => nucleic_protein
## DEBUG server => ftp.ncbi.nih.gov
## DEBUG db.fullname => alu.n : alu repeat element. alu.a : translation of alu.n repeats
## DEBUG local.files => ^alu\.(a|n).*
## DEBUG savetogalaxy.args => -u http://localhost:8080 -k c9bbb8c84e145e285f0b8b82f2cf6cd5 -l "$dbname" --lib-desc "just testing" -f "$dirversion" $datadir/$dirversion/$localrelease

So it doesn't appear to list any errors here. Leaving the edit screen and going back in, it's empty again (but the .properties file is still filled with the information). Any ideas?

[osallou]

can you connect to the database specified in global.properties in db.name (let's suppose it is biomaj ).

#> mongo biomaj
MongoDB shell version: 2.4.9
connecting to: biomaj
 > db.banks.find({name:'alu'}).pretty()
... display of the bank ...
 > exit

You should look at display result and search for the owner, example:

"properties" : {
    "owner" : "osallou",

Can you confirm that owner is mark ?

However, as you are an admin, you should have access anyway. In the development.ini do you have:

admin=mark (no whitespaces)

[osallou]

I suppose that in console you still have a 403 Forbidden

[markiskander]

Alright so I cloned and installed a fresh copy of biomaj/biomaj-watcher just to make sure I didn't just have an error somewhere in a config file. I also added admin=mark instead of mark=admin. Making a new bank seems to work just fine now, and the information is saved.

Where I'm stuck is getting one of the biomaj-data examples to work. I copied Anopheles_gambiae.properties to the /etc folder indicated in global.properties. I also copied all of the .sh files that it requires and put them in /process. I understand that editing a bank isn't really possible until you run --update on it with biomaj-cli. Makes sense. I'm stuck at these errors right now, though, when I'm trying to do --update: 2015-09-17 09:18:24,577 ERROR [root][Thread-10] PROCESS:ERROR:CONV1_multiplelink 2015-09-17 09:18:25,045 ERROR [root][Thread-11] PROCESS:ERROR:CONV2_makeblastdb 2015-09-17 09:18:25,097 ERROR [root][MainThread] Error during task postprocess 2015-09-17 09:18:25,288 ERROR [root][MainThread] Could not send email: [Errno 111] Connection refused

Error #1 and #2 log files are below: /home/mark/custombiomaj/process/multiplelinkfile.sh: 30: /home/mark/custombiomaj/process/multiplelinkfile.sh: let: not found /home/mark/custombiomaj/process/multiplelinkfile.sh: 46: /home/mark/custombiomaj/process/multiplelinkfile.sh: let: not found /home/mark/custombiomaj/process/multiplelinkfile.sh: 46: /home/mark/custombiomaj/process/multiplelinkfile.sh: let: not found [] Ce fichier existe deja! impossible de creer ce lien !

and

/home/mark/custombiomaj/process/makeblastdb.sh: line 137: makeblastdb: command not found Formatdb failed with status 127

Error #3 is just about #1 and #2. Error #4 is perhaps only a warning and I shouldn't worry about it? Or is email connection mandatory?

Thanks again

Edit: the owner was root and I'm always running everything as root anyway. And yeah, Mark is now an admin (extra buttons showing up to update and update from scratch on biomaj-watcher)

[osallou]

Email error is a warning, it is not blocking.

Example post-processes, provided by us or the community , sometimes rely on other programs that need to be installed. They may not all work "as provided" and could need some customization.

/home/mark/custombiomaj/process/makeblastdb.sh: this postprocess calls the makeblastdb program, which must be installed on your system and in your path.

/home/mark/custombiomaj/process/multiplelinkfile.sh: it could be a matter of shebang (executing with bash while being a sh/csh script for example). The "let: not found" looks like an interpretation issue. Or it could be a problem with the args of the process not matching any downloaded file. Did you check in flat directory for *.fna files ? If not are present, it could be the issue. I have not checked extensively those example files that were migrated from previous biomaj version.

I gonna try locally for this one.

[markiskander]

I'll install makeblastdb.

As for multiplelinkfile.sh, I added a #!/bin/csh and it gave me a different error about an illegal variable name. I'll try the tcsh shell and see if it works.

Thanks

Edit: seems to give an entirely different error: if: Expression Syntax. I tried bash and it seems to be working for multiplelink!

[osallou]

I have tried adding at first line

 #!/bin/bash

and it worked

I have updated on repo, for future use...

[osallou]

but consider that some may not work directly because needing other tools/config etc... there are maybe provided for help in creating processes or adaptation, they may not work "as is".

[markiskander]

One of the causes for the some of the connection refused issues I ran into was that the requirements.txt for biomaj includes the 4 packages without a version number, whereas the requirements.txt for biomaj-watcher includes the version numbers. So this resulted in some version mismatches between them. Perhaps enforce the versions in the biomaj requirements.txt?

mock==1.0.1 nose==1.3.4 pymongo==2.7.2 pycurl==7.19.5

[osallou]

which connections ? the Forbidden in the config update? versions should not be an issue, I do not think it is related but rather to your config errors in development.ini. Required dependencies are not involved (except pymongo) in biomaj-watcher interactions

[markiskander]

Not the forbidden, rather 500s when saving/updating sometimes. It gives an error "no module named connection"

[markiskander]

So far I've fixed the two previous post-process errors. One more error is coming up:

2015-09-17 10:27:47,281 INFO [root][MainThread] Log file: /home/mark/custombiomaj/log/Anopheles_gambiae/1442503667.26/Anopheles_gambiae.log Log file: /home/mark/custombiomaj/log/Anopheles_gambiae/1442503667.26/Anopheles_gambiae.log 2015-09-17 10:27:47,284 WARNI [root][MainThread] Bank:Anopheles_gambiae:Update 2015-09-17 10:27:47,285 INFO [root][MainThread] Bank:Anopheles_gambiae:Release:latest 2015-09-17 10:27:47,285 INFO [root][MainThread] Workflow:Start 2015-09-17 10:27:47,285 INFO [root][MainThread] Workflow:Start:init 2015-09-17 10:27:47,285 INFO [root][MainThread] Workflow:wf_init 2015-09-17 10:27:47,287 INFO [root][MainThread] Workflow:Skip:check 2015-09-17 10:27:47,287 INFO [root][MainThread] Workflow:Skip:depends 2015-09-17 10:27:47,287 INFO [root][MainThread] Workflow:Skip:preprocess 2015-09-17 10:27:47,288 INFO [root][MainThread] Workflow:Skip:release 2015-09-17 10:27:47,288 INFO [root][MainThread] Workflow:Skip:download 2015-09-17 10:27:47,288 INFO [root][MainThread] Workflow:Start:postprocess 2015-09-17 10:27:47,288 INFO [root][MainThread] Workflow:wf_postprocess 2015-09-17 10:27:47,288 INFO [root][MainThread] PROC:POST:BLOCK 2015-09-17 10:27:47,288 INFO [root][MainThread] PROC:POST:BLOCK:CONVERT 2015-09-17 10:27:47,290 INFO [root][Thread-1] PROC:META:RUN:CONV1 2015-09-17 10:27:47,292 INFO [root][Thread-2] PROC:META:RUN:CONV2 2015-09-17 10:27:47,292 INFO [root][Thread-1] PROC:META:SKIP:PROCESS:multiplelink 2015-09-17 10:27:47,295 INFO [root][Thread-2] PROC:META:SKIP:PROCESS:concat 2015-09-17 10:27:47,295 INFO [root][Thread-1] PROC:META:SKIP:PROCESS:linkProt 2015-09-17 10:27:47,295 INFO [root][Thread-2] PROC:META:RUN:PROCESS:makeblastdb 2015-09-17 10:27:47,296 INFO [root][Thread-2] PROCESS:RUN:CONV2_makeblastdb 2015-09-17 10:27:50,029 INFO [root][Thread-2] PROCESS:EXEC:CONV2_makeblastdb:True 2015-09-17 10:27:50,067 INFO [root][MainThread] PROC:POST:BLOCK:GALAXY 2015-09-17 10:27:50,068 ERROR [root][MainThread] Workflow:postprocessException:'NoneType' object has no attribute 'split' 2015-09-17 10:27:50,108 ERROR [root][MainThread] Error during task postprocess 2015-09-17 10:27:50,108 INFO [root][MainThread] Workflow:wf_over 2015-09-17 10:27:50,111 INFO [root][MainThread] Notify:mark 2015-09-17 10:27:50,112 INFO [root][MainThread] BANK[Anopheles_gambiae] - STATUS[False] - UPDATE[True] - REMOVE[False] - RELEASE[AgamP3] 2015-09-17 10:27:50,166 ERROR [root][MainThread] Could not send email: [Errno 111] Connection refused

Any ideas?

[osallou]

Here, error is in biomaj code Certainly a problem in property file that raise an exception. I will test locally but it will have to wait for tomorrow.

It seems it fails in biomaj/process/processfactory.py line 220

metas = self.bank.config.get(process_block.strip()+'.db.post.process').split(',')

Just to be sure, could you update file to add some debug logs

        # run each block
        logging.warn('##DEBUG#'+process_block.strip()+"#")
        metas = self.bank.config.get(process_block.strip()+'.db.post.process').split(',')
        logging.warn('##DEBUG METAS#'+str(metas)+"#")

and give me the console output of a new update?

[osallou]

By the way postprocess will certainly not work (after debugging this):

args refer to:

 /opt/biomaj/biomaj-process/biomaj2galaxy/galaxy_xxx.conf

This needs to be adapted to your path/config files.

[markiskander]

Doesn't seem to have changed the console output:

2015-09-17 11:06:06,875 INFO [root][MainThread] Log file: /home/mark/custombiomaj/log/Anopheles_gambiae/1442505966.88/Anopheles_gambiae.log 2015-09-17 11:06:06,877 WARNI [root][MainThread] Bank:Anopheles_gambiae:Update 2015-09-17 11:06:06,877 INFO [root][MainThread] Bank:Anopheles_gambiae:Release:latest 2015-09-17 11:06:06,877 INFO [root][MainThread] Workflow:Start 2015-09-17 11:06:06,877 INFO [root][MainThread] Workflow:Start:init 2015-09-17 11:06:06,878 INFO [root][MainThread] Workflow:wf_init 2015-09-17 11:06:06,878 INFO [root][MainThread] Workflow:Skip:check 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:Skip:depends 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:Skip:preprocess 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:Skip:release 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:Skip:download 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:Start:postprocess 2015-09-17 11:06:06,879 INFO [root][MainThread] Workflow:wf_postprocess 2015-09-17 11:06:06,879 INFO [root][MainThread] PROC:POST:BLOCK 2015-09-17 11:06:06,879 INFO [root][MainThread] PROC:POST:BLOCK:CONVERT 2015-09-17 11:06:06,881 INFO [root][Thread-1] PROC:META:RUN:CONV1 2015-09-17 11:06:06,881 INFO [root][Thread-1] PROC:META:SKIP:PROCESS:multiplelink 2015-09-17 11:06:06,882 INFO [root][Thread-2] PROC:META:RUN:CONV2 2015-09-17 11:06:06,882 INFO [root][Thread-1] PROC:META:SKIP:PROCESS:linkProt 2015-09-17 11:06:06,882 INFO [root][Thread-2] PROC:META:SKIP:PROCESS:concat 2015-09-17 11:06:06,883 INFO [root][Thread-2] PROC:META:SKIP:PROCESS:makeblastdb 2015-09-17 11:06:06,884 INFO [root][MainThread] PROC:POST:BLOCK:GALAXY 2015-09-17 11:06:06,884 ERROR [root][MainThread] Workflow:postprocessException:'NoneType' object has no attribute 'split' 2015-09-17 11:06:06,885 ERROR [root][MainThread] Error during task postprocess 2015-09-17 11:06:06,885 INFO [root][MainThread] Workflow:wf_over 2015-09-17 11:06:06,887 INFO [root][MainThread] Notify:mark 2015-09-17 11:06:06,887 INFO [root][MainThread] BANK[Anopheles_gambiae] - STATUS[False] - UPDATE[True] - REMOVE[False] - RELEASE[AgamP3__1] 2015-09-17 11:06:06,890 ERROR [root][MainThread] Could not send email: [Errno 111] Connection refused

[markiskander]

Error appears to be fixed after editing the .properties file accordingly.

The last thing to mention is that I ran into 500s updating alu until I made sure that requirements.txt for both biomaj and biomaj-watcher referred to the same version numbers.

According to: http://api.mongodb.org/python/current/changelog.html#mongoclient-changes It seems connection module is deprecated as of pymongo 3. The person in the following post is having an error similar to the one I was having: http://stackoverflow.com/questions/31240952/pymongo-3-0-3-importerror-no-module-named-connection

It seems when requirements.txt refers to pymongo, pymongo >3 is installed. Adding ==2.7.2 to the requirements.txt gets rid of the error.

[osallou]

ok, I gonna add requirement

Just to be sure of fix, can you remove bank , and re-update it ? You only changed arguments of process galaxy? If yes I wonder why it has disappeared

biomaj-cli.py --remove-all --bank Anopheles...
biomaj-cli.py --update --bank Anapheles...

[markiskander]

For our uses, we currently didn't actually need the galaxy block in the .properties file. So the properties file is now:

[GENERAL]

######################

Initialization

db.fullname="Genome Anopheles gambiae (NCBI)" db.name=Anopheles_gambiae db.type=genome db.formats=fasta,blast,bowtie2,bwa,2bit

protocol=ftp server=ftp.ncbi.nlm.nih.gov files.num.threads=3

release.file.compressed=false release.regexp= release.file=^[a-zA-Z]+[0-9.]+([.\w]+)$

remote.dir=/genomes/refseq/invertebrate/Anopheles_gambiae/latest_assembly_versions/ remote.files=[.\w]+/[.\w]+_assembly_structure/Primary_Assembly/assembledchromosomes/FASTA/(chr[\w]+.fna.gz)$ [.\w]+/[.\w]+(protein.faa.gz)$ remote.excluded.files=

local.files=[.\w]+$ [\w]+/[.\w]+$ dir.version=genomes/Anopheles_gambiae-ncbi offline.dir.name=biomaj/OfflineDir/genomes/Anopheles_gambiae-ncbi no.extract=false

frequency.update=0 keep.old.version=0

Post Process ### The files should be located in the projectfiles/process directory.

BLOCKS=CONVERT

CONVERT.db.post.process=CONV1,CONV2

CONV1=multiplelink,linkProt

multiplelink.name=multipleLink multiplelink.desc=Link all chromosomes multiplelink.type=link multiplelink.args='flat/*.fna' 1 'fasta' multiplelink.cluster=false multiplelink.exe=multiplelinkfile.sh

linkProt.name=linkProt linkProt.desc=Link protein file linkProt.type=link linkProt.args='flat/*protein.faa' 'fasta/protein.faa' linkProt.cluster=false linkProt.exe=linkfile.sh

CONV2=concat,makeblastdb

concat.name=concat concat.desc=concat fasta files concat.type=concat concat.args="flat/*.fna" "" "fasta/all.fasta" fasta concat.cluster=false concat.exe=concat.sh

makeblastdb.name=makeblastdb makeblastdb.desc=Index blast makeblastdb.type=index makeblastdb.args="fasta/all.fasta" "blast/" "-dbtype nucl -parse_seqids" Anopheles_gambiae-ncbi makeblastdb.cluster=false makeblastdb.exe=makeblastdb.sh

My mistake was that BLOCKS still referred to GALAXY even though the GALAXY block was not there. So that explains why GALAXY was throwing an error. Thanks for helping me fix the other two errors though!

[osallou]

no problem, do not hesitate to ask.

For general purpose questions, do not hesitate to use our google group: https://groups.google.com/forum/#!forum/biomaj

I will upload soon at update release of biomaj, containing among other things the requirement on pymongo release.