Exception: diamond error! error code: -9

[naikd0]

Hello, hope you are all having a good day. I am running some analysis of some agalactiae isolates and for the most part this was successful in its execution however I have 5 isolates that did not successfully run due to the same error.

Traceback (most recent call last): File "/home/dnaik/miniconda3/bin/bakta", line 10, in sys.exit(main()) File "/home/dnaik/miniconda3/lib/python3.9/site-packages/bakta/main.py", line 248, in main cdss_psc, cdss_pscc, cdss_not_found = psc.search(cdss_not_found)

File "/home/dnaik/miniconda3/lib/python3.9/site-packages/bakta/psc.py", line 63, in search raise Exception(f'diamond error! error code: {proc.returncode}') Exception: diamond error! error code: -9 282results (copy).zip 3917results (copy).zip 6501results (copy).zip 7359results (copy).zip 6742results (copy).zip

I have several others that I used with bakta and I had no trouble so I am wondering why these isolates are giving me the same error, any insight into this would be greatly appreciated.

Have a good day, Dowd Naik

I

[oschwengers]

Hi @naikd0, thanks for reporting. Due to your description and your logs, this error is not caused by Bakta itself but by Diamond which is used internally for sequence alignments. One common reason for this is a lack of enough RAM. Could you make sure, Diamond has at least 8 Gb of available memory? Sometimes, even more is required. To be safe and to exclude this source of error, I'd suggest to run Bakta with 12 Gb on these genomes.

[naikd0]

Good morning @oschwengers , thank you for getting back to me!

I see, it is strange though that these isolates did not have enough RAM to complete bakta but the others did. I am going to install Bakta on a computer with enough RAM, that should solve my problem. Thanks again for your speedy response, have a good day!

[oschwengers]

Yes, that's more of an educated guess than a proven source of error. I've seen various issues with Diamond in this context: some actual bugs within Bakta that we've fixed, some were due to not enough computational resources and some we couldn't reproduce and just vanished. I let this open until hearing back from you. Best regards!

[oschwengers]

HI @naikd0 , any news on this? I'll close this for now. If the issue still remains, please do not hesitate to re-open it. Best regards!

[jfy133]

Hi @oschwengers,

I just came across this exact same problem when I was testing Bakta on my laptop in a nextflow pipeline, and I have a couple of requests/suggestions:

1. It would be good to document the memory requirements (if not already, sorry if I've missed this) on the README somewhere, as the error I got in this case is rather opaque

   File "/usr/local/lib/python3.10/site-packages/bakta/main.py", line 244, in main
     cdss_psc, cdss_pscc, cdss_not_found = psc.search(cdss_not_found)
   File "/usr/local/lib/python3.10/site-packages/bakta/psc.py", line 64, in search
     raise Exception(f'diamond error! error code: {proc.returncode}')
   Exception: diamond error! error code: -9

and as someone 'trying out' the tool, it would sort of put me off using it.

2. It would also be helpful (in a nextflow context) that if Bakta hits the error with DIAMOND, that it exits with an exit status the same/similar to DIAMOND. This is because Nextflow has a system that allows us to re-try jobs with more memory (on clusters), if the tool fails with a useful exit message. Currently Bakta exits with a a general '1'

If you would be willing to consider these, I'm happy to make separate issues.

[oschwengers]

Thanks @jfy133 for bringing this up. I'll add a bit of resource requirements/recommendations to the readme.

Regarding the Diamond error code passthrough: Actually, due to Benjamin Buchfink the developer of Diamond, Diamond only uses 0 and 1 for normal and abnormal exits, respectively as stated here: https://github.com/oschwengers/bakta/issues/147#issuecomment-1269979770 So therefore, I'm rather reluctant to passthrough these error codes since I wonder if these might be somehow platform-dependent?

Currently, we also use Bakta with Nextflow very often and whenever it fails with default resources (8 cores, 16Gb memory) we double the memory resources up to 3 times. This mitigates most of these issues - not a perfect solution but it's working.

[jfy133]

updating the README would be good!

Huh ok interesting. That's a fair point about platform-dependent exit codes, and having the 'minimum' resources will likely be sufficient in this case with a limited retry on a the generic error.

I'm normally nervous about just allowing retry on any exit 1 (looking at you java :shakesfist:), but as I have confirmation from you that you do this is good enough for me :+1:

Thanks for the reply!