Python, New simplified interface: indexing(run_metrics) -> ZeroDivisionError

[sklages]

from interop import read
from interop import indexing

run_folder = '/path/to/run_folder'

run_metrics = read(run_folder)
indexing(run_metrics)

results in a ZeroDivisionError:

---------------------------------------------------------------------------

ZeroDivisionError                         Traceback (most recent call last)

<ipython-input-9-0c45127d2d2b> in <module>
     11 #index_summary(run_metrics)
     12 #summary(run_metrics)
---> 13 indexing(run_metrics)
     14 #indexing('/path/to/run_folder')

/opt/bio/anaconda3/lib/python3.8/site-packages/interop/core.py in indexing(run_metrics, per_sample, dtype, stype, **extra)
    568                             , index_info.cluster_count()
    569                             , metric.cluster_count_pf()
--> 570                             , float(index_info.cluster_count()) / metric.cluster_count_pf() * 100.0
    571                             )
    572                 k += 1

ZeroDivisionError: float division by zero

Using indexing() with the run_folder path works:

from interop import indexing

run_folder = '/path/to/run_folder'

indexing(run_folder)

Result:

array([(1., 1101., 'ACAGTG', 'Sample-xyz5-ACAGTG', 4.11300e+04, nan, nan),
       (1., 1101., 'GCCAAT', 'Sample-xyz6-GCCAAT', 3.28960e+04, nan, nan),
       (1., 1101., 'ATCACG', 'Sample-xyz1-ATCACG', 3.60270e+04, nan, nan),
       ...,
       (2., 2488., 'CCAACA', 'Sample-xyz36-CCAACA', 1.22000e+02, nan, nan),
       (2., 2488., 'CTAGCT', 'Sample-xyz38-CTAGCT', 5.00000e+00, nan, nan),
       (2., 2488., 'CTATAC', 'Sample-xyz39-CTATAC', 2.21123e+05, nan, nan)],
      dtype=[('Lane', '<f4'), ('Tile', '<f4'), ('Barcode', 'O'), ('SampleID', 'O'), ('Cluster Count', '<f4'), ('Cluster Count PF', '<f4'), ('% Demux', '<f4')])

Can you reproduce this?

Addendum

For the latter there are some values missing: Cluster Count PF and % Demux ...

That looks even weirder for some other runs:

array([(1., 1101., 'TAATCGAT-AGATCTCG', 'Sample-1', 54281., nan, nan),
       (1., 1101., 'GTAGAGAT-AGATCTCG', 'Sample-2', 65510., nan, nan),
       (1., 1101., 'TAAGGCGA-TATGCAGT', 'Sample-3', 73180., nan, nan),
       ..., (0.,    0., 0, 0,     0.,  0.,  0.),
       (0.,    0., 0, 0,     0.,  0.,  0.),
       (0.,    0., 0, 0,     0.,  0.,  0.)],
      dtype=[('Lane', '<f4'), ('Tile', '<f4'), ('Barcode', 'O'), ('SampleID', 'O'), ('Cluster Count', '<f4'), ('Cluster Count PF', '<f4'), ('% Demux', '<f4')])

Not sure what's going wrong here.

[ezralanglois]

Looks like we never used or tested loading directly from a run folder. There is a bug in valid_to_load when doing that, we are missing 'Tile' metrics.

The ZeroDivisionError is nasty little issue unique to Python. We have not encountered it yet, so did not guard against it. This is due to one or more of your tiles have zero PF.

I can fix both bugs and put in a test case for the second one. We don't have regression tests setup around this yet, so there may be more unknown issues when loading from a run folder directly.

[sklages]

okay, no worries. I am happy to test :-)

[sklages]

pip install interop still lists 1.1.22 .. am I too impatient? ;-)

[ezralanglois]

It should have been ready, it looks like the docker image failed to download and the build failed.

[ezralanglois]

This will take a little time to fix, due to recent updates to github and travis I have lost permissions to just kick off a build manually.

[ezralanglois]

It should all be working now. I need to dig in and see why docker is having so many network issues.

[sklages]

Now it worked,

Installing collected packages: interop
  Attempting uninstall: interop
    Found existing installation: interop 1.1.22
    Uninstalling interop-1.1.22:
      Successfully uninstalled interop-1.1.22
Successfully installed interop-1.1.23

Thank you!