Closed svenvarkel closed 3 years ago
Same here when trying to run a spider with arguments
212 Simply use ScrapydWeb to deploy your projects, which would generate a meaningful version for you, to be specific, the last modification time of files in your project or your egg file, like '2018-01-01T00_00_01'。
I think this would break automated deployment workflow. Or - is it possible to deploy automatically via ScrapydWeb?
I'm unable to list versions or spiders with error message:
<' not supported between instances of 'str' and 'int'
It seems that: versions can only be integers.
if you deploy your spider like this:
scrapyd-deploy -v 1
scrapyd-deploy -v 2
list versions will work ok
Hi @svenvarkel, @ohenrik , Do you use git versioning in scrapyd-client? Can you share more info like configuration for scrapyd-client scrapyd log and the full message returned by the webservice?
I hit the same bug, I used scrapyd-client deploy <my_project>
with scrapy.cfg
...
version = GIT
...
Deployment works fine with 'Statuscode 200' However, listing projects or scheduling projects causes following scrapyd stacktrace
2019-10-14T06:21:24+0000 [_GenericHTTPChannelProtocol,16,172.17.0.1] Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/twisted/web/http.py", line 2237, in allContentReceived
req.requestReceived(command, path, version)
File "/usr/local/lib/python3.7/dist-packages/twisted/web/http.py", line 937, in requestReceived
self.process()
File "/usr/local/lib/python3.7/dist-packages/twisted/web/server.py", line 217, in process
self.render(resrc)
File "/usr/local/lib/python3.7/dist-packages/twisted/web/server.py", line 284, in render
body = resrc.render(self)
--- <exception caught here> ---
File "/usr/local/lib/python3.7/dist-packages/scrapyd/webservice.py", line 21, in render
return JsonResource.render(self, txrequest).encode('utf-8')
File "/usr/local/lib/python3.7/dist-packages/scrapyd/utils.py", line 20, in render
r = resource.Resource.render(self, txrequest)
File "/usr/local/lib/python3.7/dist-packages/twisted/web/resource.py", line 265, in render
return m(request)
File "/usr/local/lib/python3.7/dist-packages/scrapyd/webservice.py", line 50, in render_POST
spiders = get_spider_list(project, version=version)
File "/usr/local/lib/python3.7/dist-packages/scrapyd/utils.py", line 134, in get_spider_list
raise RuntimeError(msg.encode('unicode_escape') if six.PY2 else msg)
builtins.RuntimeError: Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 40, in <module>
main()
File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 35, in main
with project_environment(project):
File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 16, in project_environment
version, eggfile = eggstorage.get(project, eggversion)
File "/usr/local/lib/python3.7/dist-packages/scrapyd/eggstorage.py", line 28, in get
version = self.list(project)[-1]
File "/usr/local/lib/python3.7/dist-packages/scrapyd/eggstorage.py", line 37, in list
return sorted(versions, key=LooseVersion)
File "/usr/lib/python3.7/distutils/version.py", line 52, in __lt__
c = self._cmp(other)
File "/usr/lib/python3.7/distutils/version.py", line 337, in _cmp
if self.version < other.version:
TypeError: '<' not supported between instances of 'str' and 'int'
Closing as duplicate as #212.
I'm unable to list versions or spiders with error message:
<' not supported between instances of 'str' and 'int'