fanc boundaries Intel MKL ERROR

[aminakur]

I am using fanc boundaries tool: fanc boundaries -w 1mb ,2mb 5kb.insulation \ 5kb.insulation_boundaries \

But the code fails. There is a long error message, but it boils down to: Intel MKL ERROR: Parameter 6 was incorrect on entry to DGELSD. and UnboundLocalError: local variable 'b' referenced before assignment

Could you please help me solve this?

[kaukrise]

This seems to be an issue with the Savitzky-Golay filter, which FAN-C applies to smooth and calculate derivatives of the insulation track, as indicated here:

https://stackoverflow.com/questions/65304879/intel-mkl-error-parameter-6-was-incorrect-on-entry

Can you please let me know what operating system you are using?

[aminakur]

I am running this on a Linux cluster (Red Hat distribution, I believe)

[kaukrise]

Thanks, I'll run some tests and get back to you tomorrow, it is getting late here

[aminakur]

Thank you! Looking forward.

[kaukrise]

Unfortunately I was unable to reproduce the error. I tried on two different systems, a Linux cluster (Centos 7.6) and an iMac (macOS 11.1). Can you please post the entire output of your command, including the complete error message, here? And please use the "Insert code" function. Thank you!

[kaukrise]

Digging a bit further, this seems to be an issue with your numpy installation. Please see this post: https://github.com/etal/cnvkit/issues/508#issuecomment-640369931

Are you using conda or a regular Python environment?

[aminakur]

I believe I am using regular Python environment. The output:

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: fanc in /home/ak8725/.local/lib/python3.8/site-packages (0.9.19)
Requirement already satisfied: progressbar2 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (3.53.1)
Requirement already satisfied: genomic-regions>=0.0.10 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.0.10)
Requirement already satisfied: gridmap>=0.14.0 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.15.0)
Requirement already satisfied: biopython in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (1.79)
Requirement already satisfied: intervaltree in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (3.1.0)
Requirement already satisfied: pandas>=0.15.0 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from fanc) (1.1.3)
Requirement already satisfied: pyBigWig in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.3.18)
Requirement already satisfied: cooler>=0.8.0 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.8.11)
Requirement already satisfied: PyYAML>=5.1 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from fanc) (5.3.1)
Requirement already satisfied: pytest in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (6.2.4)
Requirement already satisfied: h5py in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (3.3.0)
Requirement already satisfied: numpy>=1.16.0 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages/numpy-1.19.2-py3.8-linux-x86_64.egg (from fanc) (1.19.2)
Requirement already satisfied: scikit-learn in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.24.2)
Requirement already satisfied: scikit-image>=0.15.0 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.18.1)
Requirement already satisfied: seaborn in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.11.1)
Requirement already satisfied: msgpack-numpy>=0.4.6.1 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.4.7.1)
Requirement already satisfied: matplotlib>=3.1.0 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from fanc) (3.3.2)
Requirement already satisfied: msgpack>=1.0.0 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (1.0.2)
Requirement already satisfied: tables>=3.5.1 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (3.6.1)
Requirement already satisfied: pysam>=0.9.1 in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.16.0.1)
Requirement already satisfied: future in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.18.2)
Requirement already satisfied: pybedtools in /home/ak8725/.local/lib/python3.8/site-packages (from fanc) (0.8.2)
Requirement already satisfied: pillow in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from fanc) (8.0.1)
Requirement already satisfied: scipy in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages/scipy-1.5.2-py3.8-linux-x86_64.egg (from fanc) (1.5.2)
Requirement already satisfied: python-utils>=2.3.0 in /home/ak8725/.local/lib/python3.8/site-packages (from progressbar2->fanc) (2.5.6)
Requirement already satisfied: six in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from progressbar2->fanc) (1.15.0)
Requirement already satisfied: psutil>=2.0.0 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from gridmap>=0.14.0->fanc) (5.8.0)
Requirement already satisfied: drmaa in /home/ak8725/.local/lib/python3.8/site-packages (from gridmap>=0.14.0->fanc) (0.7.9)
Requirement already satisfied: cloudpickle in /home/ak8725/.local/lib/python3.8/site-packages (from gridmap>=0.14.0->fanc) (1.6.0)
Requirement already satisfied: pyzmq in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from gridmap>=0.14.0->fanc) (19.0.2)
Requirement already satisfied: sortedcontainers<3.0,>=2.0 in /home/ak8725/.local/lib/python3.8/site-packages (from intervaltree->fanc) (2.4.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from pandas>=0.15.0->fanc) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from pandas>=0.15.0->fanc) (2020.1)
Requirement already satisfied: multiprocess in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (0.70.12.2)
Requirement already satisfied: asciitree in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (0.3.3)
Requirement already satisfied: cytoolz<0.11 in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (0.10.1)
Requirement already satisfied: simplejson in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (3.17.2)
Requirement already satisfied: click>=7 in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (8.0.1)
Requirement already satisfied: pyfaidx in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (0.5.9.5)
Requirement already satisfied: pypairix in /home/ak8725/.local/lib/python3.8/site-packages (from cooler>=0.8.0->fanc) (0.3.7)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /home/ak8725/.local/lib/python3.8/site-packages (from pytest->fanc) (0.13.1)
Requirement already satisfied: packaging in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from pytest->fanc) (20.4)
Requirement already satisfied: iniconfig in /home/ak8725/.local/lib/python3.8/site-packages (from pytest->fanc) (1.1.1)
Requirement already satisfied: py>=1.8.2 in /home/ak8725/.local/lib/python3.8/site-packages (from pytest->fanc) (1.10.0)
Requirement already satisfied: toml in /home/ak8725/.local/lib/python3.8/site-packages (from pytest->fanc) (0.10.2)
Requirement already satisfied: attrs>=19.2.0 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from pytest->fanc) (20.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-learn->fanc) (2.1.0)
Requirement already satisfied: joblib>=0.11 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-learn->fanc) (1.0.1)
Requirement already satisfied: networkx>=2.0 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-image>=0.15.0->fanc) (2.5.1)
Requirement already satisfied: imageio>=2.3.0 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-image>=0.15.0->fanc) (2.9.0)
Requirement already satisfied: PyWavelets>=1.1.1 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-image>=0.15.0->fanc) (1.1.1)
Requirement already satisfied: tifffile>=2019.7.26 in /home/ak8725/.local/lib/python3.8/site-packages (from scikit-image>=0.15.0->fanc) (2021.6.14)
Requirement already satisfied: certifi>=2020.06.20 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from matplotlib>=3.1.0->fanc) (2020.6.20)
Requirement already satisfied: kiwisolver>=1.0.1 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from matplotlib>=3.1.0->fanc) (1.2.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from matplotlib>=3.1.0->fanc) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from matplotlib>=3.1.0->fanc) (0.10.0)
Requirement already satisfied: numexpr>=2.6.2 in /home/ak8725/.local/lib/python3.8/site-packages (from tables>=3.5.1->fanc) (2.7.3)
Requirement already satisfied: dill>=0.3.4 in /home/ak8725/.local/lib/python3.8/site-packages (from multiprocess->cooler>=0.8.0->fanc) (0.3.4)
Requirement already satisfied: toolz>=0.8.0 in /home/ak8725/.local/lib/python3.8/site-packages (from cytoolz<0.11->cooler>=0.8.0->fanc) (0.11.1)
Requirement already satisfied: setuptools>=0.7 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from pyfaidx->cooler>=0.8.0->fanc) (49.2.1)
Requirement already satisfied: decorator<5,>=4.3 in /share/apps/python/3.8.6/intel/lib/python3.8/site-packages (from networkx>=2.0->scikit-image>=0.15.0->fanc) (4.4.2)
WARNING: You are using pip version 20.2.3; however, version 21.1.3 is available.
You should consider upgrading via the '/share/apps/python/3.8.6/intel/bin/python -m pip install --upgrade pip' command.
2021-07-07 12:30:37,812 INFO FAN-C version: 0.9.19
/home/ak8725/.local/lib/python3.8/site-packages/fanc/tools/matrix.py:76: UserWarning: Warning: converting a masked element to nan.
  delta[i] = np.ma.mean(x[down_slice] - x[i]) - np.ma.mean(x[up_slice] - x[i])
2021-07-07 12:30:54,662 INFO Found 5815 raw peaks

Intel MKL ERROR: Parameter 6 was incorrect on entry to DGELSD.
Traceback (most recent call last):
  File "/home/ak8725/.local/lib/python3.8/site-packages/fanc/commands/fanc_commands.py", line 3211, in boundaries
    b = Boundaries.from_insulation_score(regions, min_score=min_score,
  File "/home/ak8725/.local/lib/python3.8/site-packages/fanc/architecture/domains.py", line 603, in from_insulation_score
    peaks = MaximaCallerDelta(index, window_size=delta_window, sub_bin_precision=sub_bin_precision)
  File "/home/ak8725/.local/lib/python3.8/site-packages/fanc/architecture/maxima_callers.py", line 86, in __init__
    self._call_peaks()
  File "/home/ak8725/.local/lib/python3.8/site-packages/fanc/architecture/maxima_callers.py", line 92, in _call_peaks
    self.delta_d1 = savgol_filter(self.delta, window_length=2*self.window_size + 1, polyorder=2, deriv=1)
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/scipy-1.5.2-py3.8-linux-x86_64.egg/scipy/signal/_savitzky_golay.py", line 346, in savgol_filter
    _fit_edges_polyfit(x, window_length, polyorder, deriv, delta, axis, y)
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/scipy-1.5.2-py3.8-linux-x86_64.egg/scipy/signal/_savitzky_golay.py", line 218, in _fit_edges_polyfit
    _fit_edge(x, 0, window_length, 0, halflen, axis,
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/scipy-1.5.2-py3.8-linux-x86_64.egg/scipy/signal/_savitzky_golay.py", line 188, in _fit_edge
    poly_coeffs = np.polyfit(np.arange(0, window_stop - window_start),
  File "<__array_function__ internals>", line 5, in polyfit
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/numpy-1.19.2-py3.8-linux-x86_64.egg/numpy/lib/polynomial.py", line 629, in polyfit
    c, resids, rank, s = lstsq(lhs, rhs, rcond)
  File "<__array_function__ internals>", line 5, in lstsq
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/numpy-1.19.2-py3.8-linux-x86_64.egg/numpy/linalg/linalg.py", line 2306, in lstsq
    x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
  File "/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/numpy-1.19.2-py3.8-linux-x86_64.egg/numpy/linalg/linalg.py", line 100, in _raise_linalgerror_lstsq
    raise LinAlgError("SVD did not converge in Linear Least Squares")
numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ak8725/.local/bin/fanc", line 127, in <module>
    Fanc()
  File "/home/ak8725/.local/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/home/ak8725/.local/lib/python3.8/site-packages/fanc/commands/fanc_commands.py", line 3218, in boundaries
    b.close()
UnboundLocalError: local variable 'b' referenced before assignment
Closing remaining open files:NAFGAM...done

[kaukrise]

Yes, this appears to be the same error as in the link I posted above. This indicates an issue with your numpy installation, which I don't know how to fix. I would suggest following the advice in that post, search for the error in conjunction with numpy, or look for help in the numpy forums.

[aminakur]

Installing numpy without the MKL did fix the issue! Thank you.