djm / python-scrapyd-api

A Python wrapper for working with Scrapyd's API.
BSD 2-Clause "Simplified" License
268 stars 32 forks source link

ModuleNotFoundError: No module named 'robot' #17

Closed LostAcapulco closed 4 years ago

LostAcapulco commented 4 years ago

I'm trying to run my spider via scrapyd_api like follow.

scrapyd = ScrapydAPI('http://localhost:6800') spider_pid = scrapyd.schedule( 'default', 'scrap_data', id=loca_id, token=local_token ) But at the moment to run my code this error shows up

ModuleNotFoundError: No module named 'robot'

I tried to load manually the path to my project and setting up the PYTHONPATH but I'm getting the same error

This is the traceback

2020-07-07T22:40:05-0500 [Launcher,32634/stderr] Traceback (most recent call last):
      File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapyd/runner.py", line 40, in <module>
        main()
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapyd/runner.py", line 37, in main
        execute()
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapy/cmdline.py", line 112, in execute
        settings = get_project_settings()
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapy/utils/project.py", line 69, in get_project_settings
        settings.setmodule(settings_module_path, priority='project')
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapy/settings/__init__.py", line 287, in setmodule
        module = import_module(module)
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'robot'
djm commented 4 years ago
      File "/Volumes/Transcend/Development/analizar/lib/python3.7/site-packages/scrapy/settings/__init__.py", line 287, in setmodule
        module = import_module(module)

This line is failing with the error ModuleNotFoundError: No module named 'robot'

Which means your Scrapy settings (or modules imported by your Scrapy settings) are trying to import a package named robot, which the Python install cannot find for whatever reason.

As we don't have a package called robot in this package; it's unlikely to be related to a bug in this project and as such I am closing the issue. Best of luck finding an answer, but this question is better suited to StackOverflow.

djm commented 4 years ago

If you'd like to paste a gist of your import, maybe I can spot something. It's quite possibly you don't have your PYTHON PATH environment variable set up correctly, such that scrapyd cannot find that robot module as it's not on the path.

On Thu, 9 Jul 2020 at 01:04, LostAcapulco notifications@github.com wrote:

Hi DJm, thanks for your answer, I know that the problem comes from the import of the module but from my perspective something is not working in scrapyd or scrapyd_api because when I run manually scrapy in the CLI this error doesn´t show up. So the error in not from my side, Thank any way

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/djm/python-scrapyd-api/issues/17#issuecomment-655819199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAVVKBDTZVX3VXLNNF5NQTR2UCRJANCNFSM4OUDWNTQ .

-- Darian Moody w: djm.org.uk