search

PaulMcInnis / JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.
MIT License
1.85k stars 215 forks source link

Failing in ubuntu OS #36

Closed byteshiva closed 4 years ago

byteshiva commented 4 years ago

Failed to install funnel on ubuntu 19.10. Am I missing something? Are there any prerequisite for funnel installation. 

$ pip3 install git+https://github.com/PaulMcInnis/JobFunnel.git

Error logs

    ERROR: Command errored out with exit status 1:
     command: /home/xyz/.asdf/installs/python/3.8.0/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-n91obe4e/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-n91obe4e/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-dbbevzm6/install-record.txt --single-version-externally-managed --compile
         cwd: /tmp/pip-install-n91obe4e/scikit-learn/
    Complete output (90 lines):
    Partial import of sklearn during the build process.
    blas_opt_info:
    blas_mkl_info:
    customize UnixCCompiler
      libraries mkl_rt not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    blis_info:
      libraries blis not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    openblas_info:
      libraries openblas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    atlas_3_10_blas_threads_info:
    Setting PTATLAS=ATLAS
      libraries tatlas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    atlas_3_10_blas_info:
      libraries satlas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    atlas_blas_threads_info:
    Setting PTATLAS=ATLAS
      libraries ptf77blas,ptcblas,atlas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    atlas_blas_info:
      libraries f77blas,cblas,atlas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    accelerate_info:
      NOT AVAILABLE

    /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/system_info.py:1896: UserWarning:
        Optimized (vendor) Blas libraries are not found.
        Falls back to netlib Blas library which has worse performance.
        A better performance should be easily gained by switching
        Blas library.
      if self._calc_info(blas):
    blas_info:
      libraries blas not found in ['/home/xyz/.asdf/installs/python/3.8.0/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/system_info.py:1896: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      if self._calc_info(blas):
    blas_src_info:
      NOT AVAILABLE

    /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/system_info.py:1896: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      if self._calc_info(blas):
      NOT AVAILABLE

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-n91obe4e/scikit-learn/setup.py", line 290, in <module>
        setup_package()
      File "/tmp/pip-install-n91obe4e/scikit-learn/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "/tmp/pip-install-n91obe4e/scikit-learn/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 66, in configuration
        config.add_subpackage('utils')
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/xyz/.asdf/installs/python/3.8.0/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-n91obe4e/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-n91obe4e/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-dbbevzm6/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
PaulMcInnis commented 4 years ago

hmm I'm not sure why you're seeing this issue, I've just tested installer working myself.

Can you give me more information about your version of ubuntu?

Additionally can you try running with python 3.6

biko-the-bird commented 4 years ago

I am also having the same issue but with different output. Python3.5 and python3.6 are installed on my system (ubuntu 16.04 lts) pip is only setup with python3.5 so that seems to be the one that JobFunnel gets run with.

Screen Shot 2020-01-05 at 5 55 43 PM

When I try install jobfunnel manually from github I encounter the same issue.

PaulMcInnis commented 4 years ago

Can you both confirm for me the results of running which pipare python 3?

justAsascha commented 4 years ago

Same on Arch Linux.

➜  / which pip
/usr/bin/pip
➜  / python --version
Python 3.8.1
➜  / pip --version
pip 19.2.3 from /usr/lib/python3.8/site-packages/pip (python 3.8)
byteshiva commented 4 years ago 
uname -a 
Linux desktop 5.3.0-26-generic #28-Ubuntu SMP  2019 x86_64 x86_64 x86_64 GNU/Linux
➜ which pip3 
/home/xyz/.asdf/shims/pip3
➜ python --version 
Python 3.8.0
➜ pip --version 
pip 19.3.1 from /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages/pip (python 3.8)
PaulMcInnis commented 4 years ago

Ok, I've replicated this issue. Its an issue with the version of scikit learn and python.

If you use python 3.7.6 it should work fine.

PaulMcInnis commented 4 years ago

Alternatively, wait until v 2.0 is merged, which fixes this issue.

edit, its been released, updating should fix this for latest python.

byteshiva commented 4 years ago

The installation went fine with v 2.0. Now Its works fine for me.

➜ pip3 install git+https://github.com/PaulMcInnis/JobFunnel.git
Collecting git+https://github.com/PaulMcInnis/JobFunnel.git
  Cloning https://github.com/PaulMcInnis/JobFunnel.git to ./pip-req-build-bmm7ykj6
  Running command git clone -q https://github.com/PaulMcInnis/JobFunnel.git /tmp/pip-req-build-bmm7ykj6
Requirement already satisfied: beautifulsoup4>=4.6.3 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (4.8.1)
Requirement already satisfied: lxml>=4.2.4 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (4.4.2)
Requirement already satisfied: requests>=2.19.1 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (2.22.0)
Requirement already satisfied: python-dateutil>=2.8.0 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (2.8.1)
Requirement already satisfied: PyYAML>=5.1 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (5.2)
Collecting scikit-learn>=0.21.2
  Downloading https://files.pythonhosted.org/packages/db/5d/dddb8b82ee573df2ccaacb210e1df56daec33dd62aa637005dbc7b889eea/scikit_learn-0.22.1-cp38-cp38-manylinux1_x86_64.whl (7.0MB)
     |████████████████████████████████| 7.0MB 142kB/s 
Collecting nltk>=3.4.1
  Downloading https://files.pythonhosted.org/packages/f6/1d/d925cfb4f324ede997f6d47bea4d9babba51b49e87a767c170b77005889d/nltk-3.4.5.zip (1.5MB)
     |████████████████████████████████| 1.5MB 27kB/s 
Requirement already satisfied: scipy>=1.4.1 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from JobFunnel==2.0.0) (1.4.1)
Requirement already satisfied: soupsieve>=1.2 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from beautifulsoup4>=4.6.3->JobFunnel==2.0.0) (1.9.4)
Requirement already satisfied: idna<2.9,>=2.5 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from requests>=2.19.1->JobFunnel==2.0.0) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from requests>=2.19.1->JobFunnel==2.0.0) (2019.9.11)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from requests>=2.19.1->JobFunnel==2.0.0) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from requests>=2.19.1->JobFunnel==2.0.0) (1.25.7)
Requirement already satisfied: six>=1.5 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from python-dateutil>=2.8.0->JobFunnel==2.0.0) (1.12.0)
Requirement already satisfied: numpy>=1.11.0 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from scikit-learn>=0.21.2->JobFunnel==2.0.0) (1.18.0)
Requirement already satisfied: joblib>=0.11 in /home/xyz/.asdf/installs/python/3.8.0/lib/python3.8/site-packages (from scikit-learn>=0.21.2->JobFunnel==2.0.0) (0.14.1)
Installing collected packages: scikit-learn, nltk, JobFunnel
    Running setup.py install for nltk ... done
    Running setup.py install for JobFunnel ... done
Successfully installed JobFunnel-2.0.0 nltk-3.4.5 scikit-learn-0.22.1
/tmp took 1m34s 
➜ funnel --help
usage: CLI options take precedence over settings in the yaml fileempty arguments are replaced by settings in the default yaml file
       [-h] [-s SETTINGS] [-o OUTPUT_PATH] [-kw [KEYWORDS [KEYWORDS ...]]] [-r] [-c] [-d DELAY] [-md MIN_DELAY] [--fun {constant,linear,sigmoid}]
       [--log_level {critical,error,warning,info,debug,notset}] [--similar] [--no_scrape] [--no_delay] [--recover] [--save_dup]