fake-name / xA-Scraper

69 stars 8 forks source link

A few questions #68

Closed WashingtonSite closed 4 years ago

WashingtonSite commented 4 years ago

1) Does this still work?

2) trying to run "run-all.py" nets me with

Traceback (most recent call last): File "run-all.py", line 1, in <module> import plugins.scrapers.fa.faScrape ImportError: No module named plugins.scrapers.fa.faScrape

fake-name commented 4 years ago

Does this still work?

Yes

trying to run "run-all.py" nets me with

How are you running the file? Your environment is broken in some manner.

If you're bootstrapping a new system, you need run.sh, which will configure a virtualenv for you and set up the database, etc...

Unless you need to manage stuff manually, try to use the provided shell scripts.

WashingtonSite commented 4 years ago

I did everything manually through terminal and pip. I tried via the requirements.txt but cherry and some others were not working and the script would quit.

sh run.sh Venv exists. Activating! run.sh: 22: run.sh: source: not found

fake-name commented 4 years ago

but cherry and some others were not working and the script would quit.

Uh, if some of the dependencies are broken, maybe that's your problem?

What python are you using? On what platform?

sh run.sh Venv exists. Activating! run.sh: 22: run.sh: source: not found

Your venv folder is present, but doesn't contain an actual virtualenv. Delete the folder, and try running run.sh again.

WashingtonSite commented 4 years ago

I'm running python 3.6 on ubuntu desktop 18.04. I deleted venvfolder and ran run.sh once more but

$ sh run.sh
No Venv! Checking dependencies are installed.
[sudo] password for user:
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version (12.4ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 116 not upgraded.
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'zlib1g-dev' instead of 'libz-dev'
zlib1g-dev is already the newest version (1:1.2.11.dfsg-0ubuntu2).
libxml2 is already the newest version (2.9.4+dfsg1-6.1ubuntu1.2).
libxslt1-dev is already the newest version (1.1.29-5ubuntu0.2).
python3-dev is already the newest version (3.6.7-1~18.04).
0 upgraded, 0 newly installed, 0 to remove and 116 not upgraded.
Creating venv.
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/home/user/.wget-hsts'. HSTS will be disabled.
--2019-11-15 23:45:43--  https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 199.232.36.175, 2a04:4e42:31::175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|199.232.36.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1775835 (1.7M) [text/x-python]
Saving to: ‘get-pip.py’

get-pip.py                    100%[=================================================>]   1.69M  5.76MB/s    in 0.3s

2019-11-15 23:45:44 (5.76 MB/s) - ‘get-pip.py’ saved [1775835/1775835]

Collecting pip
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/d9/de/554b6310ac87c5b921bc45634b07b11394fe63bc4cb5176f5240addf18ab/setuptools-41.6.0-py2.py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-19.3.1 setuptools-41.6.0 wheel-0.33.6
run.sh: 34: run.sh: source: not found
fake-name commented 4 years ago

run.sh: 34: run.sh: source: not found`

Your shell is.... odd. Have you replaced bash with something else?

source is a bash builtin that you're somehow missing.

WashingtonSite commented 4 years ago

No, I wouldn't know how to remove or fix that. I managed to get around that problem using bash run.sh. I appreciate your help narrowing down the issue.

ModuleNotFoundError: No module named 'cherrypy' I have installed cherrypy and am working on trying to fix this error. I'll leave another comment later.

fake-name commented 4 years ago

ModuleNotFoundError: No module named 'cherrypy'

Hmmm, it's in the requirements file. I think your environment is still weird.

Frankly, the fact that #!/usr/bin/env bash is resolving to something that's not bash makes me think you have something more fundamental broken. Basically, anything that relies on shell scripts as part of it's setup process is likely broken on your system.

Did you change your shell in some odd manner? What do you get if you manually run /usr/bin/env bash?

WashingtonSite commented 4 years ago

I've not changed anything in regards to the shell. I get "cannot execute binary file".

fake-name commented 4 years ago

I dunno what to tell you other then your system is broken somehow. Using /usr/bin/env is the generally correct way to run <binary> in a environment aware manner.

One thing I can think of. What does file run.sh tell you? I wonder if maybe you got a checkout of the project that has windows line endings. That could bean you're actually trying to run bash\r, which indeed wouldn't exist.

WashingtonSite commented 4 years ago

That'd be pretty lame haha. It tells me: run.sh: Bourne-Again shell script, ASCII text executable.

fake-name commented 4 years ago

The fact that it doesn't say anything about windows line endings means you're probably fine for that.

On my ubuntu 18.04 install, /usr/bin/env bash just drops me into bash. So whatever changed how that behaves is your problem source, here.

WashingtonSite commented 4 years ago

Ok well thank you for the help, I'll mess around with another clean installation.