issue with libpng12 on ubuntu v18

[StefanoLonardi]

I have used ALLMAPS dozen of times with no problems. We have recently upgraded our server to the UBUNTU 18.04.1 LTS, and now ALLMAPS fails when it is time to create the PDFs. Here is the error message in log.txt

....
20:30:52 [agp] Write object chr10 to `map.chr.fasta`
20:30:53 [agp] Write object chr11 to `map.chr.fasta`
20:30:53 [agp] Target fasta written to `map.chr.fasta`.
20:30:53 [base] Load file `map.chr.agp`
20:30:53 [base] Load file `input.fasta.sizes`
20:30:53 [allmaps] Write unplaced AGP to `map.unplaced.agp`.
20:30:53 [base] Load file `map.unplaced.agp`
20:30:53 [base] Load file `input.fasta`
20:30:59 [agp] Target fasta written to `map.unplaced.fasta`.
20:30:59 [base] cat map.chr.agp map.unplaced.agp >map.agp
20:30:59 [base] cat map.chr.fasta map.unplaced.fasta >map.fasta
20:31:00 [base] Load file `map.agp`
20:31:00 [base] Load file `input.fasta.sizes`
20:31:00 [base] Load file `map.agp`
20:31:00 [base] Load file `input.fasta.sizes`
20:31:00 [base] faSize -detailed /24-2/home/stelo/dovetail_long_bean_1_610/ALLMAPS/map.fasta >map.fasta.siz
es
faSize: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directory
20:31:00 [base] Load file `map.fasta.sizes`
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1871, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 827, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 96, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1518, in path
    build([inputbed, fastafile])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1638, in build
    fromagp([combined_agp, scaffolds, combined_fasta])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/formats/chain.py", line 145, in fromagp
    objectsizes = Sizes(objectfasta).mapping
  File "/usr/local/lib/python2.7/dist-packages/jcvi/formats/sizes.py", line 61, in __init__
    ctgs, sizes = zip(*sizes)
ValueError: need more than 0 values to unpack

It seems that ALLMAPS is looking for libpng12, but the newest version of UBUNTU has version 16. I tried to create a symbolic link from v16 to v12, but then ALLMAPS fails with another error.

Stefano

[tanghaibao]

@StefanoLonardi

There is one external binary that ALLMAPS calls - faSize (from Kent tools) which was compiled on an older machine. As a result, the call failed and the output map.fasta.sizes is empty. I think this binary is optional now, but you still need to remove faSize from your path along with the empty map.fasta.sizes in the current directory. An alternative solution, is to re-compile faSize from Kent tools so that it links against the correct lib version.

Haibao

[StefanoLonardi]

I resolved the issue related to 'faSize', but now I am getting a different error message.

...
16:13:10 [__init__] CACHEDIR=/home/stelo/.cache/matplotlib
16:13:10 [font_manager] Using fontManager instance from /home/stelo/.cache/matplotlib/fontList.json
16:13:10 [__init__] backend agg version v2.2
16:13:10 [base] Load file `map.bed`
16:13:14 [allmaps] Map contains 146581 markers in 110 linkage groups.
16:13:17 [allmaps] Retained 144,278 of 146,581 (98.4%) clean markers.
16:13:17 [base] Load file `weights.txt`
16:13:17 [base] Imported 10 records from `weights.txt`.
16:13:17 [allmaps] Map weights: [('27B', 1), ('WC', 1), ('27I', 1), ('SV', 1), ('NF', 1), ('46I', 1), ('27U', 1), ('ZZ', 1), ('TI', 1), ('5I', 1)]
16:13:17 [allmaps] Linkage function: double-linkage
16:13:17 [allmaps] Partition LGs based on 27B
16:13:47 [base] Load file `input.fasta.sizes`
16:13:47 [allmaps] Working on 27I-1|27B-1|27U-1|46I-1|5I-1|NF-1|SV-1|TI-1|WC-1|ZZ-1 ...
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1871, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 827, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 96, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1484, in path
    ngen=ngen, npop=npop, cpus=cpus, seed=seed)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 142, in __init__
    self.prepare_linkage_groups()  # populate all data
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 249, in prepare_linkage_groups
    function=self.function, linkage=self.linkage)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 92, in __init__
    self.oo[k] = get_rho(physical_to_cm)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 791, in get_rho
    rho = spearmanr(x, y)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/algorithms/formula.py", line 112, in spearmanr
    return 1 - distancematrix((x, y), dist="s")[1][0]
  File "/usr/local/lib/python2.7/dist-packages/Bio/Cluster/__init__.py", line 500, in distancematrix
    _cluster.distancematrix(data, mask, weight, transpose, dist, matrix)
RuntimeError: unexpected format.

[StefanoLonardi]

It seems the same error mentioned in #110 and #97. What's the solution?

[StefanoLonardi]

stelo@H4$ python -c "import Bio; print(Bio.__version__)"
1.72
stelo@H4:$ python -c "import numpy as np; print(np.__version__)"
1.15.4

[tanghaibao]

@StefanoLonardi

This issue is related to Spearman's rho calculation and its dependency on an older version of BioPython. Since this has caused issues for other users (per #97 and #110), I changed the code so that now it calls scipy.stats.spearmanr instead. The related code changes is in commit 9447c71.

You'll need to pull these changes, of course. I have also updated the PyPI version so you could just upgrade jcvi.

pip install -U jcvi

Haibao

[StefanoLonardi]

Haibao, thank you!