kylebgorman
commented
4 years ago Thanks for the report.
I can fix the compilation flags used for OpenFst upstream (not Pynini though), but cannot replicate the issue locally. (I didn't try the Docker version.)
On Wed, Jan 8, 2020 at 11:23 AM Pierre-Andre Noel notifications@github.com wrote:
Version 2.0.9.post2 changed https://github.com/kylebgorman/pynini/commit/76178c3e25d6570b425b2a30f188cf835f7a6f23#diff-2eeaed663bd0d25b7e608891384b7298L24-R24 one of gcc's hard-coded flags from -std=c++11 to -std=c++17. However, the latest OpenFst (i.e., 1.7.5) still uses the hard-coded -std=c++11 flag, which induces incompatible symbols in the *.so.
Example:
FROM python:3.7-buster
OpenFst
ENV FST_VERSION "1.7.5"ENV FST_DOWNLOAD_PREFIX "http://www.openfst.org/twiki/pub/FST/FstDownload" RUN cd /tmp \ && wget -q ${FST_DOWNLOAD_PREFIX}/openfst-${FST_VERSION}.tar.gz \ && tar -zxf openfst-${FST_VERSION}.tar.gz \ && rm openfst-${FST_VERSION}.tar.gz RUN cd /tmp/openfst-${FST_VERSION} \
&& mv configure configure.bad \ # Uncomment this and following for workaround
# && cat configure.bad | sed 's/-std=c++11/-std=c++17/' > configure \ # && rm configure.bad \ # && chmod +x configure \ && ./configure --enable-grm \ && make --jobs=4 \ && make install \ && rm -rd /tmp/openfst-${FST_VERSION}RUN ldconfig
Pynini
ENV PYNINI_VERSION "2.0.9.post2"ENV PYNINI_DOWNLOAD_PREFIX "http://www.opengrm.org/twiki/pub/GRM/PyniniDownload" RUN cd /tmp \ && wget -q ${PYNINI_DOWNLOAD_PREFIX}/pynini-${PYNINI_VERSION}.tar.gz \ && tar -zxf pynini-${PYNINI_VERSION}.tar.gz \ && rm pynini-${PYNINI_VERSION}.tar.gz RUN mkdir /tools \ && mv /tmp/pynini-${PYNINI_VERSION} /tools/pynini \ && cd /tools/pynini \ && python setup.py install
Import testRUN python -c "import pynini"
From a folder containing the above Dockerfile, calling docker build . fails on the last line with the error message:
Step 11/11 : RUN python -c "import pynini" ---> Running in 369a30cc34d8 Traceback (most recent call last): File "
", line 1, in ImportError: /usr/local/lib/python3.7/site-packages/pynini-2.0.9.post2-py3.7-linux-x86_64.egg/pynini.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZNK3fst8internal14DenseSymbolMap4FindESt17basic_string_viewIcSt11char_traitsIcEE The command '/bin/sh -c python -c "import pynini"' returned a non-zero code: 1 In short, Pynini expects the symbol fst::internal::DenseSymbolMap::Find(std::basic_string_view<char, std::char_traits
>) const but the OpenFst .so instead has fst::internal::DenseSymbolMap::Find(std::__cxx11::basic_string<char, std::char_traits , std::allocator > const&) const. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/19?email_source=notifications&email_token=AABG4OOJOAGJWZHILQTL4WTQ4X4ZVA5CNFSM4KEK5RBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IE2A5XQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OMRVPHUE2PBEZWJKGLQ4X4ZVANCNFSM4KEK5RBA .
PierreAndreNoel
lumpidu
Version 2.0.9.post2 changed one of gcc's hard-coded flags from
-std=c++11to-std=c++17. However, the latest OpenFst (i.e., 1.7.5) still uses the hard-coded-std=c++11flag, which induces incompatible symbols in the*.so.Example:
From a folder containing the above
Dockerfile, callingdocker build .fails on the last line with the error message:In short, Pynini expects the symbol
fst::internal::DenseSymbolMap::Find(std::basic_string_view<char, std::char_traits<char> >) constbut the OpenFst.soinstead hasfst::internal::DenseSymbolMap::Find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const.