search

scrapinghub / scrapinghub-entrypoint-scrapy

Scrapy entrypoint for Scrapinghub job runner
BSD 3-Clause "New" or "Revised" License
25 stars 16 forks source link

Running scripts is not longer working on Scrapycloud #84

Closed VMRuiz closed 1 month ago

VMRuiz commented 4 months ago 
Traceback (most recent call last):
  File "/app/python/lib/python3.11/site-packages/sh_scrapy/crawl.py", line 162, in _run_usercode
    _run(args, settings)
  File "/app/python/lib/python3.11/site-packages/sh_scrapy/crawl.py", line 112, in _run
    _run_pkgscript(args)
  File "/app/python/lib/python3.11/site-packages/sh_scrapy/crawl.py", line 142, in _run_pkgscript
    d.run_script(scriptname, {'__name__': '__main__'})
    ^^^^^^^^^^^^
AttributeError: 'PathDistribution' object has no attribute 'run_script'

The reason is this change

https://github.com/scrapinghub/scrapinghub-entrypoint-scrapy/blob/f31e458493a8108d59e0e4a7961ee7b4c44881f9/sh_scrapy/crawl.py#L142

It looks like PathDistribution from importlib doesn't implements run_script method.

VMRuiz commented 1 month ago

Hello @wRAR , Was this solved by https://github.com/scrapinghub/scrapinghub-entrypoint-scrapy/pull/85?

wRAR commented 1 month ago

I think so.