Get the install process to run without sudo

[gabrielStanovsky]

All packages requiring sudo (spaCy, nltk), should be moved to the prerequisites section of the readme and should be installed separately by the users.

[gabrielStanovsky]

Works properly on my machine. @judithek , @ypuzikov : can you please try installing again and let me know how it goes?

[gabrielStanovsky]

BTW, if you run this on a "contaminated" machine, there may be some root owned leftovers from running with sudo privileges in previous runs. This happened to me with a gdrive local file (getting permission error on token_v2.json). If this happens, you can fix this by running:

sudo chown <your-username> ~/.gdrive/token_v2.json

[judithek]

./scripts/install_annotator.sh does not run without sudo in my system

log ends with:

...

running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-4325.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

[gabrielStanovsky]

I suspect that this might be related to installing on the global python, or from files changed by root in previous tries of the installation.

Is it possible for you to try with virtualenv?

Thanks!

[judithek]

will try with virtualenv

[judithek]

testing in virtualenv now I have an issue with installing nltk in the virtualenv, nltk does not work.

here is the log for installing nltk in virtualenv:

(UNIF) eckle@desktop-155:~/unified-factuality$ pip install -U nltk
Collecting nltk
/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading nltk-3.2.3.tar.gz (1.2MB)
    100% |████████████████████████████████| 1.2MB 1.2MB/s 
Requirement already up-to-date: six in ./UNIF/lib/python2.7/site-packages (from nltk)
Building wheels for collected packages: nltk
  Running setup.py bdist_wheel for nltk ... done
  Stored in directory: /home/local/UKP/eckle/.cache/pip/wheels/91/06/de/1c95e3a8786fd3ae93415babb98cd13e13770570fcf5caa557
Successfully built nltk
Installing collected packages: nltk
Successfully installed nltk-3.2.3

and here is the error log when trying to import nltk:

(UNIF) eckle@desktop-155:~/unified-factuality$ UNIF/bin/python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nltk
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/nltk/__init__.py", line 128, in <module>
    from nltk.chunk import *
  File "/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/nltk/chunk/__init__.py", line 157, in <module>
    from nltk.chunk.api import ChunkParserI
  File "/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/nltk/chunk/api.py", line 13, in <module>
    from nltk.parse import ParserI
  File "/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/nltk/parse/__init__.py", line 81, in <module>
    from nltk.parse.corenlp import CoreNLPParser, CoreNLPDependencyParser
  File "/home/local/UKP/eckle/unified-factuality/UNIF/local/lib/python2.7/site-packages/nltk/parse/corenlp.py", line 17, in <module>
    import requests
ImportError: No module named requests

any thoughts?

[judithek]

I installed virtualenv-15.1.0

[gabrielStanovsky]

I tried replicating the error, but importing nltk succeeded. The only thing I can think of is that you might invoke a different python than that pip is using. Can you try running which python; which pip , to see whether they both are located in the same place?

[judithek]

I get

(UNIF) eckle@desktop-155:~/unified-factuality$ which python /home/local/UKP/eckle/unified-factuality/UNIF/bin/python (UNIF) eckle@desktop-155:~/unified-factuality$ which pip /home/local/UKP/eckle/unified-factuality/UNIF/bin/pip

[judithek]

nltk is also available inside virtualenv:

(UNIF) eckle@desktop-155:~/unified-factuality$ pip list DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning. appdirs (1.4.3) nltk (3.2.3) numpy (1.12.1) packaging (16.8) pip (9.0.1) pyparsing (2.2.0) setuptools (35.0.2) six (1.10.0) wheel (0.29.0)

[judithek]

solved - I had to simply install the requests module

[judithek]

everything has been successful up to the point where a new terminal is needed.

how to open a new terminal in the same virtualenv (i.e. keeping all the installed packages)? when I opened a new terminal (new tab or new window) the source command did not recognize the virtualenv created before

[gabrielStanovsky]

To "activate" an existing virtualenv you need to run

source <path-to-env>/bin/activate

Running this will use the environment's python, with all the packages installed in it up to that point.

[gabrielStanovsky]

So please run that in the new terminal, before calling the servers.

[judithek]

I did exactly that, but pip list results in

eckle@desktop-155:~/unified-factuality/src$ source UNIF/bin/activate                                                                                                            
(UNIF) eckle@desktop-155:~/unified-factuality/src$ pip list                                                                                                                     
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.                                                                                                                                            
appdirs (1.4.3)                                                                                                                                                                 
packaging (16.8)                                                                                                                                                                
pip (9.0.1)                                                                                                                                                                     
pyparsing (2.2.0)                                                                                                                                                               
setuptools (35.0.2)                                                                                                                                                             
six (1.10.0)                                                                                                                                                                    
wheel (0.29.0)                                                                                                                                                                  

whereas pip list in the original terminal has

(UNIF) eckle@desktop-155:~/unified-factuality/src$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
appdirs (1.4.3)
beautifulsoup4 (4.6.0)
BerkeleyInterface (0.2)
bs4 (0.0.1)
cheroot (5.5.0)
CherryPy (10.2.1)
coverage (3.7.1)
cymem (1.31.2)
cytoolz (0.8.2)
dill (0.2.6)
docopt (0.6.2)
en-core-web-sm (1.2.0)
ftfy (4.4.2)
graphviz (0.4.10)
html5lib (0.999999999)
JPype (0.5.4.2)
lxml (3.7.3)
murmurhash (0.26.4)
nltk (3.2)
nose (1.3.0)
numpy (1.12.1)
packaging (16.8)
pathlib (1.0.1)
pip (9.0.1)
plac (0.9.6)
portend (1.8)
preshed (1.0.0)
props (1.0)
pydot (1.2.3)
pyparsing (2.2.0)
PySocks (1.6.7)
python-dateutil (2.2)
python-graph-core (1.8.2)
python-graph-dot (1.8.2)
pytz (2017.2)
regex (2017.4.5)
requests (2.13.0)
setuptools (35.0.2)
six (1.10.0)
spacy (1.8.2)
svg-stack (0.0.1)
tempora (1.6.1)
termcolor (1.1.0)
thinc (6.5.2)
toolz (0.8.2)
tqdm (4.11.2)
ujson (1.35)
Unidecode (0.4.20)
wcwidth (0.1.7)
webencodings (0.5.1)
wheel (0.29.0)
wrapt (1.10.10)

[gabrielStanovsky]

That's really weird... I guess you also activated in the first terminal, before installing, right?

[judithek]

yes

[gabrielStanovsky]

A workaround can be to run the servers in the original shell in the background (using & flag). This will mess the output a little bit, since you'll see both servers printing. But if you redirect the factuality output into a file its should print ok, e.g.,

echo "Don refused to pay his taxes" | ./scripts/annotate_factuality.sh > ./output.txt

(after both servers are up)

[judithek]

(sorry for the confusion) everything runs fine now

(explanation for the virtualenv problem: I first called source in the wrong subdir which is why I created another one with the same name in a subdir ....)

so this was a successful installation using virtualenv on ubuntu 14.04

[gabrielStanovsky]

Great, no sudo required :+1: