Inverting intergenomic files and blast error

[jdobry-lab]

I have a dataset that was working fine on my old computer. I got a new computer and set everything up but now I keep having the these errors. I have not changed the datasets and did not experience these before. I did see in another case the Blast errors could be due to an old version of Diamond, but I have updated the version and still have the same errors. Please help because my manuscript for this data is under review.

gpar <- init_genespace(

• genomeIDs = c("Vacscaf178","Podarcis","Sceloporus", "Anolis", "Zootoca","Thamnophis", "Chrysemys", "platypus", "Gallus"),
• speciesIDs = c("Vacscaf178","Podarcis","Sceloporus", "Anolis", "Zootoca","Thamnophis", "Chrysemys", "platypus", "Gallus"),
• versionIDs = c("Vacscaf178","Podarcis","Sceloporus", "Anolis", "Zootoca","Thamnophis", "Chrysemys", "platypus", "Gallus"),
• ploidy = rep(1,9),
• diamondMode = "fast",
• orthofinderMethod = "fast",
• wd = runwd,
• nCores = 4,
• minPepLen = 50,
• gffString = "gff",
• pepString = "pep",
• path2orthofinder = "/Users/opt/anaconda3/envs/orthofinder/bin/orthofinder",
• path2mcscanx = "/Users/MCScanX-master",
• path2diamond = "/Users/opt/anaconda3/envs/orthofinder/bin/diamond",
• rawGenomeDir = file.path(runwd, "rawGenomes")) Initializing GENESPACE run checking genomeIDs ... PASS (Vacscaf178, Podarcis, Sceloporus, Anolis, Zootoca, Thamnophis, Chrysemys, platypus, Gallus) checking outgroup ... PASS (no outgroup) checking ploidy ... PASS (1, 1, 1, 1, 1, 1, 1, 1, 1) checking the number of parallel processes ... PASS (4) Verbosity ... PASS (TRUE) minPepLen ... PASS (50) checking working directory ... PASS (/Users/Documents/testGenespace) checking parsed gff files ... PASS (/Users/Documents/testGenespace/gff) checking parsed peptide files ... PASS (/Users/Documents/testGenespace/peptide) Checking dependencies and 3rd party installations MCScanX installation ... Warning: 'length(x) = 14 > 1' in coercion to 'logical(1)'PASS (/Users/MCScanX-master/MCScanX_h) Orthofinder installation ... Warning: 'length(x) = 58 > 1' in coercion to 'logical(1)'PASS (/Users/opt/anaconda3/envs/orthofinder/bin/orthofinder) OrthoFinder method ... Warning: 'length(x) = 58 > 1' in coercion to 'logical(1)'PASS - (fast) Orthofinder in block method ... Warning: 'length(x) = 58 > 1' in coercion to 'logical(1)'PASS (FALSE) diamond mode ... PASS (--fast) GENESPACE run successfully initialized results will be stored in /Users/Documents/testGenespace/results

  gpar$paths$diamondCall <- "/Users/opt/anaconda3/envs/orthofinder/bin/diamond"

  path <- "/Users/anaconda3/envs/orthofinder/bin/diamond"

  suppressWarnings(system2(path, "help", stdout = TRUE, stderr = TRUE))

  gpar <- run_orthofinder(

• gsParam = gpar) Synteny Parameters have not been set! Setting to defaults Running 'draft' a.k.a 'fast' genespace orthofinder method ############################################################ NOTE This method should only be used for: (1) closely related diploid species, (2) visualization/genome QC purposes, or (3) inferring orthogroups WITHIN syntenic regions ############################################################ Running 1/45 (Chrysemys vs. Chrysemys) Running 2/45 (Chrysemys vs. Podarcis) Running 3/45 (Podarcis vs. Podarcis) Running 4/45 (Chrysemys vs. Zootoca) Running 5/45 (Podarcis vs. Zootoca) Running 6/45 (Chrysemys vs. Sceloporus) Running 7/45 (Podarcis vs. Sceloporus) Running 8/45 (Chrysemys vs. Anolis) Running 9/45 (Podarcis vs. Anolis) Running 10/45 (Zootoca vs. Zootoca) Running 11/45 (Chrysemys vs. Thamnophis) Running 12/45 (Zootoca vs. Sceloporus) Running 13/45 (Podarcis vs. Thamnophis) Running 14/45 (Sceloporus vs. Sceloporus) Running 15/45 (Chrysemys vs. platypus) Running 16/45 (Zootoca vs. Anolis) Running 17/45 (Chrysemys vs. Gallus) Running 18/45 (Podarcis vs. platypus) Running 19/45 (Podarcis vs. Gallus) Running 20/45 (Sceloporus vs. Anolis) Running 21/45 (Zootoca vs. Thamnophis) Running 22/45 (Sceloporus vs. Thamnophis) Running 23/45 (Anolis vs. Anolis) Running 24/45 (Zootoca vs. platypus) Running 25/45 (Zootoca vs. Gallus) Running 26/45 (Anolis vs. Thamnophis) Running 27/45 (Sceloporus vs. platypus) Running 28/45 (Sceloporus vs. Gallus) Running 29/45 (Thamnophis vs. Thamnophis) Running 30/45 (Anolis vs. platypus) Running 31/45 (Anolis vs. Gallus) Running 32/45 (Thamnophis vs. platypus) Running 33/45 (Thamnophis vs. Gallus) Running 34/45 (platypus vs. platypus) Running 35/45 (platypus vs. Gallus) Running 36/45 (Gallus vs. Gallus) Running 37/45 (Chrysemys vs. Vacscaf178) Running 38/45 (Podarcis vs. Vacscaf178) Running 39/45 (Zootoca vs. Vacscaf178) Running 40/45 (Sceloporus vs. Vacscaf178) Running 41/45 (Anolis vs. Vacscaf178) Running 42/45 (Thamnophis vs. Vacscaf178) Running 43/45 (platypus vs. Vacscaf178) Running 44/45 (Gallus vs. Vacscaf178) Running 45/45 (Vacscaf178 vs. Vacscaf178) Done! Inverting intergenomic files ... Warning: Detected 9 column names but the data has 12 columns (i.e. invalid file). Added 3 extra default column names at the end.Warning: Detected 11 column names but the data has 12 columns (i.e. invalid file). Added 1 extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.Warning: Detected 9 column names but the data has 12 columns (i.e. invalid file). Added 3 extra default column names at the end.Warning: Detected 1 column names but the data has 12 columns (i.e. invalid file). Added 11 extra default column names at the end.Warning: Detected 6 column names but the data has 12 columns (i.e. invalid file). Added 6 extra default column names at the end.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 8 column names but the data has 12 columns (i.e. invalid file). Added 4 extra default column names at the end.Warning: Detected 11 column names but the data has 12 columns (i.e. invalid file). Added 1 extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.Warning: Detected 10 column names but the data has 12 columns (i.e. invalid file). Added 2 extra default column names at the end.Warning: Detected 7 column names but the data has 12 columns (i.e. invalid file). Added 5 extra default column names at the end.Warning: Detected 7 column names but the data has 12 columns (i.e. invalid file). Added 5 extra default column names at the end.Warning: Detected 3 column names but the data has 12 columns (i.e. invalid file). Added 9 extra default column names at the end.Warning: Detected 1 column names but the data has 12 columns (i.e. invalid file). Added 11 extra default column names at the end.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 11 column names but the data has 12 columns (i.e. invalid file). Added 1 extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.Warning: Detected 11 column names but the data has 12 columns (i.e. invalid file). Added 1 extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.Warning: Detected 11 column names but the data has 12 columns (i.e. invalid file). Added 1 extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 10 column names but the data has 12 columns (i.e. invalid file). Added 2 extra default column names at the end.Warning: Detected 5 column names but the data has 12 columns (i.e. invalid file). Added 7 extra default column names at the end.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 6 column names but the data has 12 columns (i.e. invalid file). Added 6 extra default column names at the end.Warning: Detected 4 column names but the data has 12 columns (i.e. invalid file). Added 8 extra default column names at the end.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 2 column names but the data has 12 columns (i.e. invalid file). Added 10 extra default column names at the end.Warning: Detected 9 column names but the data has 12 columns (i.e. invalid file). Added 3 extra default column names at the end.Warning: Detected 10 column names but the data has 12 columns (i.e. invalid file). Added 2 extra default column names at the end.Warning: Detected 10 column names but the data has 12 columns (i.e. invalid file). Added 2 extra default column names at the end.Warning: Detected 4 column names but the data has 12 columns (i.e. invalid file). Added 8 extra default column names at the end.Warning: Detected 8 column names but the data has 12 columns (i.e. invalid file). Added 4 extra default column names at the end.Warning: Detected 5 column names but the data has 12 columns (i.e. invalid file). Added 7 extra default column names at the end.Warning: Detected 7 column names but the data has 12 columns (i.e. invalid file). Added 5 extra default column names at the end.Warning: Detected 5 column names but the data has 12 columns (i.e. invalid file). Added 7 extra default column names at the end.Done! Running orthofinder -og on pre-computed blast: Warning: running command ''/Users/opt/anaconda3/envs/orthofinder/bin/orthofinder' -b /Users/Documents/testGenespace/orthofinder -t 4 -a 1 -X -og 2>&1' had status 1

  OrthoFinder version 2.5.4 Copyright (C) 2014 David Emms

  2023-01-28 19:36:59 : Starting OrthoFinder 2.5.4 4 thread(s) for highly parallel tasks (BLAST searches etc.) 1 thread(s) for OrthoFinder algorithm

  Checking required programs are installed

  Test can run "mcl -h" - ok Using previously calculated BLAST results in /Users/Documents/testGenespace/orthofinder/

  Running OrthoFinder algorithm

  2023-01-28 19:36:59 : Initial processing of each species ERROR: Blast00.txt is corrupted ERROR: Error processing files Blast0* Malformatted line in /Users/Documents/testGenespace/orthofinder/Blast0_0.txt Offending line was:

  Process Process-2: Traceback (most recent call last): File "/Users/opt/anaconda3/envs/orthofinder/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/Users/opt/anaconda3/envs/orthofinder/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, self._kwargs) File "/Users/opt/anaconda3/envs/orthofinder/bin/scripts_of/main.py", line 529, in Worker_ProcessBlastHits WaterfallMethod.ProcessBlastHits(args, d_pickle=d_pickle, qDoubleBlast=qDoubleBlast) File "/Users/opt/anaconda3/envs/orthofinder/bin/scripts_of/main.py", line 516, in ProcessBlastHits Bij = blast_file_processor.GetBLAST6Scores(seqsInfo, blastDir_list, seqsInfo.speciesToUse[iSpecies], seqsInfo.speciesToUse[jSpecies], qDoubleBlast=qDoubleBlast) File "/Users/opt/anaconda3/envs/orthofinder/bin/scripts_of/blast_file_processor.py", line 65, in GetBLAST6Scores for row in blastreader: _csv.Error: line contains NUL ERROR: An error occurred, please review the error messages*** they may contain useful information about the problem.

[jtlovell]

This error happens when you don't have the right orthofinder/diamond install combination. My guess is that the diamond install is not v2. GENESPACE V1 (coming soon ... at /dev now) will do this error catch up front.

[jdobry-lab]

What do I need to do to get the right orthofinder/diamond install? I tried:

conda update diamond

but it didn't seem to work. What am I missing?

Thanks, J

[jtlovell]

What version of diamond and orthofinder are you running? They should be at least v2.5.4 (orthofinder) and v2.0.14.152 (diamond). You should specify these versions when installing if you are getting older versions.

[jdobry-lab]

Thanks, I was able to update diamond its all good, until I try to inspect the pangenome. I get the following error:

Error in query_pangenome(pg = pg, refChrom = "1", startOrder = 1, endOrder = 5) : can't find the needed column names in pg; was this produced by pangenome?

[jtlovell]

Good to hear. Sorry about that error - query_genespace is not ready for primetime and will be dropped in the next release in lieu of separate query_ functions for hits, pangenome, blks and synPos files. V1.1.2, which will be at /dev next week will have these features.

[jtlovell]

FYI. GENESPACE v1.1.3 is at /dev. Thanks for the help here. Querying utilities and position highlights in riparian plots are now in there.