Zymo-Research / figaro

An efficient and objective tool for optimizing microbiome rRNA gene trimming parameters
GNU General Public License v3.0
80 stars 25 forks source link

Value error setting an array with a sequence Fastq - 16s #55

Open jayramr opened 1 year ago

jayramr commented 1 year ago

Hi,

We are getting below error with the Miseq fastq data as per below

figaro -i test -o figaro_out -a 295 -f 18 -r 19
Forward read length: 295
Reverse read length: 295
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/expectedErrorCurve.py", line 97, in calculateAverageExpectedError
    percentileExpectedError = makeExpectedErrorPercentileArrayForFastq(fastq.filePath, self.subsample, self.percentile, self.primerLength)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/expectedErrorCurve.py", line 109, in makeExpectedErrorPercentileArrayForFastq
    expectedErrorMatrix = fastqAnalysis.buildExpectedErrorMatrix(path, subsample=subsample, leftTrim=primerLength)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/fastqAnalysis.py", line 37, in buildExpectedErrorMatrix
    return numpy.array(expectedErrorMatrix, dataType, order='F')
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (36340,) + inhomogeneous part.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/gencore/conda3/envs/figaro/bin/figaro", line 8, in <module>
    sys.exit(main())
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/figaro.py", line 210, in main
    resultTable, forwardCurve, reverseCurve = trimParameterPrediction.performAnalysisLite(parameters.inputDirectory.value, parameters.minimumCombinedReadLength.value, subsample =  parameters.subsample.value, percentile = parameters.percentile.value, forwardPrimerLength=parameters.forwardPrimerLength.value, reversePrimerLength=parameters.reversePrimerLength.value, namingStandardAlias=fileNamingStandard)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/trimParameterPrediction.py", line 453, in performAnalysisLite
    forwardCurve, reverseCurve = expectedErrorCurve.calculateExpectedErrorCurvesForFastqList(fastqList, subsample=subsample, percentile=percentile, makePNG=makeExpectedErrorPlots, forwardPrimerLength=forwardPrimerLength, reversePrimerLength=reversePrimerLength)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/expectedErrorCurve.py", line 175, in calculateExpectedErrorCurvesForFastqList
    forwardExpectedErrorArray = makeExpectedErrorPercentileArrayForFastqList(forwardFastqs, subsample, percentile, forwardPrimerLength)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/expectedErrorCurve.py", line 122, in makeExpectedErrorPercentileArrayForFastqList
    expectedErrorReturns = easyMultiprocessing.parallelProcessRunner(parallelAgent.calculateAverageExpectedError, fastqList)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/site-packages/figaro/easyMultiprocessing.py", line 68, in parallelProcessRunner
    return mapper(processor, itemsToProcess, chunkSize)
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/scratch/gencore/conda3/envs/figaro/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (36340,) + inhomogeneous part.

Please advise.

Thanks jay