Issue Importing on Ubuntu/Python3

[misingnoglic]

Hello,

I'm trying to host a web-app using verovio, and I've created a droplet on DigitalOcean to try to host the app. The droplet runs Ubuntu 16.04.2

I've successfully installed verovio on python3.6 using the installation steps in the wiki (and tweaking the memory of the droplet). However, when I try to import verovio, I get this large error:

root@polyphony:~/verovio/python# python3.6
Python 3.6.1 (default, Mar 22 2017, 10:05:53)
[GCC 5.4.1 20170304] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import verovio
Traceback (most recent call last):
  File "/root/verovio/python/verovio.py", line 18, in swig_import_helper
    fp, pathname, description = imp.find_module('_verovio', [dirname(__file__)])
  File "/usr/lib/python3.6/imp.py", line 296, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named '_verovio'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/verovio/python/verovio.py", line 28, in <module>
    _verovio = swig_import_helper()
  File "/root/verovio/python/verovio.py", line 20, in swig_import_helper
    import _verovio
ImportError: /usr/local/lib/python3.6/dist-packages/verovio-0.9.12-py3.6-linux-x86_64.egg/_verovio.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3hum17Tool_musicxml2hum7convertERSoRN4pugi12xml_documentE
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "/root/verovio/python/verovio.py", line 18, in swig_import_helper
    fp, pathname, description = imp.find_module('_verovio', [dirname(__file__)])
  File "/usr/lib/python3.6/imp.py", line 296, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named '_verovio'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/verovio/python/verovio.py", line 28, in <module>
    _verovio = swig_import_helper()
  File "/root/verovio/python/verovio.py", line 20, in swig_import_helper
    import _verovio
ImportError: /usr/local/lib/python3.6/dist-packages/verovio-0.9.12-py3.6-linux-x86_64.egg/_verovio.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3hum17Tool_musicxml2hum7convertERSoRN4pugi12xml_documentE
>>>

Any idea what a solution to this could be? Thank you!

[craigsapp]

The problem is generated by this line:

https://github.com/rism-ch/verovio/blob/5b39471bbceb28990a36a5b7c37d4061b5fcac8c/src/toolkit.cpp#L469

The Tool_musicxml2hum::convert function is being used here. The error messages imply that the compiler found the file tool-musicxml2hum.h:

https://github.com/rism-ch/verovio/blob/master/include/hum/tool-musicxml2hum.h

But that the source-code file:

https://github.com/rism-ch/verovio/blob/master/src/hum/tool-musicxml2hum.cpp

was not included in the compile process.

The solution would involve adding tool-musicxml2hum.cpp to the list of files to compile.

Notice in the file:

https://github.com/rism-ch/verovio/blob/master/python/setup.py#L34

That only ../src/hum/humlib.cpp is include. Instead, all *.cpp files in the same directory should be include at this point in the python setup script.

Compare to the emscripten setup script, where all *.cpp files in src/hum are added to the compile list:

https://github.com/rism-ch/verovio/blob/master/emscripten/buildToolkit#L236

[misingnoglic]

I added the line to setup.py to compile all the files in glob("../src/hum/*.cpp"), but then I get this error when installing.

/root/verovio/python/../src/hum/humlib.cpp:20497: multiple definition of `hum::Tool_filter::run(hum::HumdrumFile&, std::ostream&)'
build/temp.linux-x86_64-3.6/../src/hum/humlib.o:/root/verovio/python/../src/hum/humlib.cpp:20497: first defined here
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

[craigsapp]

I think that means that you also need to remove the original line:

             '../src/hum/humlib.cpp',

Since it is now included twice.

[misingnoglic]

Thank you, the import now works!

[craigsapp]

You could submit a patch for the updated file setup.py (or copy it into this thread and I can update it).

[misingnoglic]

Maybe related - but I'm now getting a segmentation fault when trying to convert an MEI file to an svg:

Python 3.6.1 (default, Mar 22 2017, 10:05:53)
[GCC 5.4.1 20170304] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import verovio
>>> tk = verovio.toolkit(False)
>>> tk.setNoLayout(True)
>>> tk.setFormat("mei")
True
>>> tk.loadFile("mei_files/Fauv/plange_MENSURAL.mei")
True
>>> tk.renderToSvg(1)
Segmentation fault (core dumped)

Any idea how I Can figure out why? Let me know if this should be its own issue.

[craigsapp]

It is possible but not likely, since the previously discussed code is related to MusicXML import, and you are loading an MEI file. One possibility might be related to loading the musical fonts, with updated ones needing to be installed. (The last time you installed verovio was in February, and the fonts have been updated since then).

Another possibility is that it is specific to the file you are trying to typeset. Try the following minimal MEI file and see if there is also a segmentation fault (which would make it more likely related to font installation problems):

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="http://music-encoding.org/schema/3.0.0/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="http://music-encoding.org/schema/3.0.0/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="3.0.0">
    <meiHead>
        <fileDesc>
            <titleStmt>
                <title />
            </titleStmt>
            <pubStmt />
        </fileDesc>
    </meiHead>
    <music>
        <body>
            <mdiv>
                <score>
                    <scoreDef>
                        <staffGrp>
                            <staffDef n="1" clef.shape="G" clef.line="2" lines="5" />
                        </staffGrp>
                    </scoreDef>
                    <section>
                        <measure n="1" right="end">
                            <staff n="1">
                                <layer n="1">
                                    <note dur="1" oct="4" pname="c" accid.ges="n" />
                                </layer>
                            </staff>
                        </measure>
                    </section>
                </score>
            </mdiv>
        </body>
    </music>
</mei>

The output should be:

[misingnoglic]

When I try to render that mei file, I get the segmentation fault. However, I just installed the library yesterday on the digital ocean box, so it's not a problem with an old installation.

root@polyphony:~/test# nano test.mei 
root@polyphony:~/test# python3.6
Python 3.6.1 (default, Mar 22 2017, 10:05:53)
[GCC 5.4.1 20170304] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import verovio
>>> tk = verovio.toolkit(False)
>>> tk.setNoLayout(True)
>>> tk.setFormat("mei")
True
>>> tk.loadFile("test.mei")
True
>>> svg_string = tk.renderToSvg(1)
Segmentation fault (core dumped)

[craigsapp]

But it would be good to figure out why the new version is causing a segmentation fault...

One thing to try is to disable the Humdrum code when compiling for python, and see if the segmentation fault remains or not.

Such as adding:

    EXTRA_COMPILE_ARGS += ['-DNO_HUMDRUM_SUPPORT']

to setup.py as mentioned in issue #490.

[misingnoglic]

Even when adding that line, rebuilding and recompiling, I get the segfault. (Same code/error as before, so I won't recopy it).

Any other ideas?

[lpugin]

Do you have write access in the output directory?

On May 26, 2017 5:14 PM, "Arya Boudaie" notifications@github.com wrote:

Even when adding that line, rebuilding and recompiling, I get the segfault. (Same code/error as before, so I won't recopy it).

Any other ideas?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rism-ch/verovio/issues/603#issuecomment-304309370, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqFBMQMdTSY-TrjZ2TICcfYykrU9_QFks5r9uxqgaJpZM4NnDvx .

[misingnoglic]

Sorry - which output directory? Either way, I should (the VM is mine).

[lpugin]

Try

tk = verovio.toolkit()

I suspect you will have an error because of the fonts not found

[lpugin]

By doing

tk = verovio.toolkit(False)

You skip the initialisation of the font directory, which is an option for when you do want to set it explicitly once the toolkit it initialised. But you have to do it, with

tk.setResoucePath("/path/to/the/font/directory")

[misingnoglic]

Thank you! I forgot about that fix I did when running the code on my laptop.

The last issue I'm having with the ubuntu server is that the lyrics on the MEI files don't appear to be showing up. Here's a sample MEI file: http://www.cs.brandeis.edu//~arya/sicut_MENSURAL.mei

And this is the code I'm running:

import verovio
tk = verovio.toolkit(False)
tk.setResourcePath(r"/root/verovio/data")
tk.setNoLayout(True)
tk.setFormat("mei")
tk.loadFile("sicut_MENSURAL.mei")
tk.renderToSvgFile( "page.svg", 1 )

And this is the SVG I'm getting: http://www.cs.brandeis.edu//~arya/page.svg

If I run the similar script on my laptop (which has an older version of verovio on it), just changing the font directory, I get this:

http://www.cs.brandeis.edu//~arya/page_laptop.svg

Any idea why the words wouldn't appear? Thank you again for helping me with this (and let me know if you want me to open a new issue instead).

[ahankinson]

In Python, r"/root/verovio/data" is a regex string, so you probably don't need that.

Does /root/verovio/data contain the necessary files on the server machine?

[misingnoglic]

I just had it as the raw string to make sure nothing would be escaped - but for this directory it didn't matter too much.

Here are the files - I didn't delete any of them to my knowledge so it should be fine:

(polyphonynev) arya@polyphony:~/verovio$ cd data
(polyphonynev) arya@polyphony:~/verovio/data$ ls
Bravura      Gootville      Leipzig      text
Bravura.xml  Gootville.xml  Leipzig.xml  woff.xml
(polyphonynev) arya@polyphony:~/verovio/data$ cd text/
(polyphonynev) arya@polyphony:~/verovio/data/text$ ls
Times-bold-italic.xml  Times-bold.xml  Times-italic.xml  Times.xml

It looks the same as my windows machine, where it works:

 Directory of C:\Users\misin\OneDrive\Documents\GitHub\verovio\data

05/26/2017  02:15 AM    <DIR>          .
05/26/2017  02:15 AM    <DIR>          ..
05/26/2017  02:15 AM    <DIR>          Bravura
05/26/2017  02:15 AM            20,239 Bravura.xml
05/26/2017  02:15 AM    <DIR>          Gootville
05/26/2017  02:15 AM            13,677 Gootville.xml
05/26/2017  02:15 AM    <DIR>          Leipzig
05/26/2017  02:15 AM            17,409 Leipzig.xml
05/26/2017  02:15 AM    <DIR>          text
05/26/2017  02:15 AM             7,540 woff.xml
               4 File(s)         58,865 bytes
               6 Dir(s)  118,326,185,984 bytes free

C:\Users\misin\OneDrive\Documents\GitHub\verovio\data>cd text

C:\Users\misin\OneDrive\Documents\GitHub\verovio\data\text>ls

C:\Users\misin\OneDrive\Documents\GitHub\verovio\data\text>dir
 Volume in drive C is Local Disk
 Volume Serial Number is 2254-B0CF

 Directory of C:\Users\misin\OneDrive\Documents\GitHub\verovio\data\text

05/26/2017  02:15 AM    <DIR>          .
05/26/2017  02:15 AM    <DIR>          ..
05/26/2017  02:15 AM            12,599 Times-bold-italic.xml
05/26/2017  02:15 AM            12,783 Times-bold.xml
05/26/2017  02:15 AM            12,506 Times-italic.xml
05/26/2017  02:15 AM            12,661 Times.xml
               4 File(s)         50,549 bytes
               2 Dir(s)  118,326,185,984 bytes free

[ahankinson]

on the ubuntu machine what is the output of pwd when you are in the Veorvio data directory?

[misingnoglic]

It's: /home/arya/verovio/data

This is different from the root path I was using (I can test it using the root user if you'd like), but it still doesn't work when I set the working directory to this.

Script:

import verovio
tk = verovio.toolkit(False)
tk.setResourcePath("/home/arya/verovio/data")
tk.setNoLayout(True)
tk.setFormat("mei")
tk.loadFile("sicut_MENSURAL.mei")
tk.renderToSvgFile( "page.svg", 1 )

SVG: http://www.cs.brandeis.edu//~arya/page2.svg

[lpugin]

It looks that this might be a bug in the alignment of lyrics in non-measured music. Are you sure you are running the same version of Verovio on your laptop?

[lpugin]

The issue with lyrics not showing up in mensural music is fixed (https://github.com/rism-ch/verovio/commit/27780bb927ac6a2a4ea9fa620f29df434c93f9e0). There is still a mis-placement of the ledger lines. I will look at it but we can close this issue

[misingnoglic]

No - I have an older version of Verovio on my laptop (I didn't want to upgrade it on my laptop and then not have it working on either machine).

Should I install the develop branch of verovio for now?

[misingnoglic]

I installed the dev version of verovio and the lyrics still aren't showing up.

[craigsapp]

Review issue #591 where text was missing because it was actually being displayed in a very small size due to a CSS setting.

[misingnoglic]

Do you see anything like that here? http://174.138.49.237/compositions/31/

[craigsapp]

Pasting the MEI data http://174.138.49.237/media/common_mei/sicut_7Fkt3Aa.mei into VHV http://verovio.humdrum.org I can see text. displayed:

But looking at the SVG on your webpage, the text is missing from the SVG. So either verovio is not converting it properly (possibly because it cannot find the necessary font resources), or you are somehow extracting the text from the SVG (which does not seem likely).

The SVG code for the voice names also looks strange with 0px size, but they are still visible (maybe due to CSS resizing):

                                <text x="1286" y="1850" text-anchor="end" font-size="0px">
                    <tspan font-size="405px" class="text">triplum</tspan>
                </text>
                <text x="1286" y="3800" text-anchor="end" font-size="0px">
                    <tspan font-size="405px" class="text">motetus</tspan>
                </text>
                <text x="1286" y="5665" text-anchor="end" font-size="0px">
                    <tspan font-size="405px" class="text">tenor</tspan>
                </text>

This is also what the font size is set to when viewing in VHV (and might be the problem in issue #591 where setting the font-size in CSS to 100% causes the text to disappear).

---

Note that sicut does not have a dash between "si" and "cut"

Your encoding:

        <syl xml:id="m-58" wordpos="i">Si</syl>

Verovio needs this encoding to display the dash:

    <syl xml:id="m-58" con="d" wordpos="i">Si</syl>

Perhaps it would be better for verovio to assume @con="d" when @wordpos is "i" or "m"...

---

[misingnoglic]

Do you know how I could troubleshoot the conversion to see what's going wrong. I dont know why it wouldn't be able to find the fonts, they're all in that directory.

[craigsapp]

Is there a way of viewing the error/warning output from verovio in python? That might have a message helping to figure out the problem. In VHV I can open the console to display such messages. Here is an example of the messages that print when viewing the test data:

[misingnoglic]

When I run the code just on the REPL, it doesn't give me any errors:

(polyphonynev) arya@polyphony:~/measuring_polyphony_django/media/mensural_mei$ python
Python 3.6.1 (default, Mar 22 2017, 10:05:53)
[GCC 5.4.1 20170304] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import verovio
>>> tk = verovio.toolkit(False)
>>> tk.setResourcePath("/home/arya/verovio/data")
True
>>> tk.setNoLayout(True)
>>> tk.setFormat("mei")
True
>>> tk.loadFile("sicut_MENSURAL.mei")
True
>>> tk.renderToSvgFile( "page.svg", 1 )
True
>>> quit()

But the SVG has no lyrics still:

http://174.138.49.237/media/mensural_mei/page.svg

[lpugin]

What version are you running? Have you updated it?

[misingnoglic]

I'm running the latest dev version, pulled from here (with git checkout -b develop): https://github.com/rism-ch/verovio/commits/develop

verovio==0.9.12

[lpugin]

Please update and make sure you see

>>> import verovio
>>> tk = verovio.toolkit(False)
>>> tk.getVersion()
'1.0.1-dev-7ed264e'
>>> 

[misingnoglic]

Right now I have '1.0.0-b2194a3-dirty' - I'll try the installation again.

[misingnoglic]

I must have checked out the wrong branch. It's working now. Thank you so much for helping me solve this. http://174.138.49.237/compositions/15/

Out of curiosity - what's the preferred way of upgrading your verovio installation? Just doing a git pull doesn't work for me when building again, so I'm forced to delete the repository, reclone it, and re build it. Also - should I generally be sticking with the develop branch?

[rettinghaus]

You should use git remote add upstream https://github.com/rism-ch/verovio.git. Then just git fetch upstream and merge it into your branch. See here: https://help.github.com/articles/syncing-a-fork/

[pedro-psb]

I'm on Ubuntu 18.10 and I've a simliar problem to @misingnoglic 's first issue. I've successfully installed swig and built the python binding following wiki instructions (from current git), but I get an error when I try to import. I guess the proposed fix on setup.py was commited already, but I have a sligtly different error:

Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import verovio
Traceback (most recent call last):
  File "/home/pedro/sys-and-soft/verovio/verovio/bindings/python/verovio.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_verovio'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pedro/sys-and-soft/verovio/verovio/bindings/python/verovio.py", line 17, in <module>
    _verovio = swig_import_helper()
  File "/home/pedro/sys-and-soft/verovio/verovio/bindings/python/verovio.py", line 16, in swig_import_helper
    return importlib.import_module('_verovio')
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_verovio'