kylebgorman / pynini

Read-only mirror of Pynini
http://pynini.opengrm.org
Apache License 2.0
120 stars 26 forks source link

Installation issues #16

Closed ykl7 closed 4 years ago

ykl7 commented 4 years ago

Hi,

I installed OpenFST using the following command: ./configure --prefix=${HOME}/.local/ --enable-grm && make && make install. I also added it to my $PATH. Other OpenFST functions are working fine.

When I installed pynini (in a different directory) using pip install -e git+git@github.com:kylebgorman/pynini.git#egg=pynini, I get the following error:

running develop
    running egg_info
    writing pynini.egg-info/PKG-INFO
    writing dependency_links to pynini.egg-info/dependency_links.txt
    writing top-level names to pynini.egg-info/top_level.txt
    reading manifest file 'pynini.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pynini.egg-info/SOURCES.txt'
    running build_ext
    building 'pywrapfst' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -B /home/ykumar/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ykumar/anaconda3/include/python3.7m -c src/pywrapfst.cc -o build/temp.linux-x86_64-3.7/src/pywrapfst.o -std=c++11 -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    src/pywrapfst.cc:625:22: fatal error: fst/util.h: No such file or directory
     #include <fst/util.h>
                          ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

System specs are: Debian 4.9.82, gcc 6.3.0, Cython 0.28.5, Python 3.7.4. What am I doing wrong? What should I do instead?

kylebgorman commented 4 years ago

Good day,

The prefix you're using for installing OpenFst is apparently not in your include path when you're running the pip install bit. You can either:

Ideal stolen from here: https://stackoverflow.com/questions/10858934/how-might-one-specify-or-add-a-directory-to-the-python-h-search-path-during-a-mo

On Thu, Oct 31, 2019 at 4:17 PM Yash Kumar Lal notifications@github.com wrote:

Hi,

I installed OpenFST using the following command: ./configure --prefix=${HOME}/.local/ --enable-grm && make && make install. I also added it to my $PATH. Other OpenFST functions are working fine.

When I installed pynini (in a different directory) using pip install -e git+git@github.com:kylebgorman/pynini.git#egg=pynini, I get the following error:

running develop

running egg_info

writing pynini.egg-info/PKG-INFO

writing dependency_links to pynini.egg-info/dependency_links.txt

writing top-level names to pynini.egg-info/top_level.txt

reading manifest file 'pynini.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

writing manifest file 'pynini.egg-info/SOURCES.txt'

running build_ext

building 'pywrapfst' extension

creating build

creating build/temp.linux-x86_64-3.7

creating build/temp.linux-x86_64-3.7/src

gcc -pthread -B /home/ykumar/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ykumar/anaconda3/include/python3.7m -c src/pywrapfst.cc -o build/temp.linux-x86_64-3.7/src/pywrapfst.o -std=c++11 -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char

cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++

src/pywrapfst.cc:625:22: fatal error: fst/util.h: No such file or directory

 #include <fst/util.h>

                      ^

compilation terminated.

error: command 'gcc' failed with exit status 1

System specs are: Debian 4.9.82, gcc 6.3.0, Cython 0.28.5, Python 3.7.4. What am I doing wrong? What should I do instead?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/16?email_source=notifications&email_token=AABG4OIHDIWX2QTDBXSMSJTQRM4M5A5CNFSM4JHR43V2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HV5GVVA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OMS4KCNYGIRBREBJZDQRM4M5ANCNFSM4JHR43VQ .

ykl7 commented 4 years ago

Hello,

I tried the method you suggested, but I get the following:

error: option --with-includepath not recognized

It looks like setup.py does not recognize the argument

kylebgorman commented 4 years ago

Try this, then (I just read the documentation):

python setup.py build_ext --include-dirs=$HOME && python setup.py test

Or if this doesn't work, install OpenFst in /usr/local or ask your administrator to do so for you.

ykl7 commented 4 years ago

Just to confirm, $HOME is the path to OpenFST? Or a particular folder in it?

kylebgorman commented 4 years ago

No that was just an example. Instead of $HOME use wherever you installed OpenFst. Here I think it'll be like:

--include-dirs="${HOME}/.local"

or perhaps

 --include-dirs="${HOME}/.local/include"

Not sure which yet---as I said I can't test this locally without nuking my own install.

ykl7 commented 4 years ago

So, it does find the files now (I think so). It's a whole different error log:

running build_ext
building 'pywrapfst' extension
gcc -pthread -B /home/ykumar/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/export/c02/ykumar/openfst-1.7.4/src/include -I/home/ykumar/anaconda3/include/python3.7m -c src/pywrapfst.cc -o build/temp.linux-x86_64-3.7/src/pywrapfst.o -std=c++11 -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /export/c02/ykumar/openfst-1.7.4/src/include/fst/fst.h:29:0,
                 from /export/c02/ykumar/openfst-1.7.4/src/include/fst/expanded-fst.h:19,
                 from /export/c02/ykumar/openfst-1.7.4/src/include/fst/fstlib.h:28,
                 from src/pywrapfst.cc:626:
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘virtual int64 fst::internal::SymbolTableImpl::GetNthKey(ssize_t) const’:
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:241:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (pos < 0 || pos >= symbols_.Size()) return kNoSymbol;
                    ~~~~^~~~~~~~~~~~~~~~~~
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘bool fst::SymbolTableIterator::Done() const’:
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:475:36: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   bool Done() const { return (pos_ == nsymbols_); }
                               ~~~~~^~~~~~~~~~~~
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘void fst::SymbolTableIterator::Next()’:
/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:486:14: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (pos_ < nsymbols_) key_ = table_.GetNthKey(pos_);
         ~~~~~^~~~~~~~~~~
g++ -pthread -shared -B /home/ykumar/anaconda3/compiler_compat -L/home/ykumar/anaconda3/lib -Wl,-rpath=/home/ykumar/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/pywrapfst.o -lfstfarscript -lfstfar -lfstscript -lfst -lm -ldl -o build/lib.linux-x86_64-3.7/pywrapfst.cpython-37m-x86_64-linux-gnu.so
/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstfarscript
/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstfar
/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstscript
/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfst
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1

Also, it has to point to the /include directory.

kylebgorman commented 4 years ago

(None of those warnings are interesting---they're just warnings.)

It found the headers it needed, but can't link at the end, so you need to do something similar with the path to libraries (yet another reason to install globally)...judging from what you said upthread:

python setup.py build_ext  --include-dirs="${HOME}/.local/include"

--library-dirs="${HOME}/.local/lib" install

K

On Thu, Oct 31, 2019 at 5:08 PM Yash Kumar Lal notifications@github.com wrote:

So, it does find the files now (I think so). It's a whole different error log:

running build_ext

building 'pywrapfst' extension

gcc -pthread -B /home/ykumar/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/export/c02/ykumar/openfst-1.7.4/src/include -I/home/ykumar/anaconda3/include/python3.7m -c src/pywrapfst.cc -o build/temp.linux-x86_64-3.7/src/pywrapfst.o -std=c++11 -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char

cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++

In file included from /export/c02/ykumar/openfst-1.7.4/src/include/fst/fst.h:29:0,

             from /export/c02/ykumar/openfst-1.7.4/src/include/fst/expanded-fst.h:19,

             from /export/c02/ykumar/openfst-1.7.4/src/include/fst/fstlib.h:28,

             from src/pywrapfst.cc:626:

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘virtual int64 fst::internal::SymbolTableImpl::GetNthKey(ssize_t) const’:

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:241:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

 if (pos < 0 || pos >= symbols_.Size()) return kNoSymbol;

                ~~~~^~~~~~~~~~~~~~~~~~

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘bool fst::SymbolTableIterator::Done() const’:

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:475:36: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

bool Done() const { return (pos == nsymbols); }

                           ~~~~~^~~~~~~~~~~~

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h: In member function ‘void fst::SymbolTableIterator::Next()’:

/export/c02/ykumar/openfst-1.7.4/src/include/fst/symbol-table.h:486:14: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

 if (pos_ < nsymbols_) key_ = table_.GetNthKey(pos_);

     ~~~~~^~~~~~~~~~~

g++ -pthread -shared -B /home/ykumar/anaconda3/compiler_compat -L/home/ykumar/anaconda3/lib -Wl,-rpath=/home/ykumar/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/pywrapfst.o -lfstfarscript -lfstfar -lfstscript -lfst -lm -ldl -o build/lib.linux-x86_64-3.7/pywrapfst.cpython-37m-x86_64-linux-gnu.so

/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstfarscript

/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstfar

/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfstscript

/home/ykumar/anaconda3/compiler_compat/ld: cannot find -lfst

collect2: error: ld returned 1 exit status

error: command 'g++' failed with exit status 1

Also, it has to point to the /include directory.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/16?email_source=notifications&email_token=AABG4OL7COJZESOSUQNPAT3QRNCMZA5CNFSM4JHR43V2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECZICMY#issuecomment-548569395, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OKJ4TB5R46D55ZYUVLQRNCMZANCNFSM4JHR43VQ .

ykl7 commented 4 years ago

I'm still getting the same error. I'll try to find the exact path of the libraries inside the openfst folder.

(I asked the admin, and they refused to install it globally)