scrapy / scrapyd

A service daemon to run Scrapy spiders
https://scrapyd.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
2.97k stars 569 forks source link

<' not supported between instances of 'str' and 'int' when listing versions #281

Closed svenvarkel closed 3 years ago

svenvarkel commented 6 years ago

I'm unable to list versions or spiders with error message:

<' not supported between instances of 'str' and 'int'

ohenrik commented 6 years ago

Same here when trying to run a spider with arguments

svenvarkel commented 5 years ago

212 Simply use ScrapydWeb to deploy your projects, which would generate a meaningful version for you, to be specific, the last modification time of files in your project or your egg file, like '2018-01-01T00_00_01'。

I think this would break automated deployment workflow. Or - is it possible to deploy automatically via ScrapydWeb?

shimachao commented 5 years ago

I'm unable to list versions or spiders with error message:

<' not supported between instances of 'str' and 'int'

It seems that: versions can only be integers. if you deploy your spider like this: scrapyd-deploy -v 1 scrapyd-deploy -v 2

list versions will work ok

Digenis commented 5 years ago

Hi @svenvarkel, @ohenrik , Do you use git versioning in scrapyd-client? Can you share more info like configuration for scrapyd-client scrapyd log and the full message returned by the webservice?

wildgarden commented 5 years ago

I hit the same bug, I used scrapyd-client deploy <my_project> with scrapy.cfg

...
version = GIT
...

Deployment works fine with 'Statuscode 200' However, listing projects or scheduling projects causes following scrapyd stacktrace

2019-10-14T06:21:24+0000 [_GenericHTTPChannelProtocol,16,172.17.0.1] Unhandled Error                                                                                                                        
        Traceback (most recent call last):                                                                                                                                                                  
          File "/usr/local/lib/python3.7/dist-packages/twisted/web/http.py", line 2237, in allContentReceived                                                                                               
            req.requestReceived(command, path, version)                                                                                                                                                     
          File "/usr/local/lib/python3.7/dist-packages/twisted/web/http.py", line 937, in requestReceived                                                                                                   
            self.process()                                                                                                                                                                                  
          File "/usr/local/lib/python3.7/dist-packages/twisted/web/server.py", line 217, in process                                                                                                         
            self.render(resrc)                                                                                                                                                                              
          File "/usr/local/lib/python3.7/dist-packages/twisted/web/server.py", line 284, in render                                                                                                          
            body = resrc.render(self)                                                                                                                                                                       
        --- <exception caught here> ---                                                                                                                                                                     
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/webservice.py", line 21, in render
            return JsonResource.render(self, txrequest).encode('utf-8')
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/utils.py", line 20, in render
            r = resource.Resource.render(self, txrequest)
          File "/usr/local/lib/python3.7/dist-packages/twisted/web/resource.py", line 265, in render
            return m(request)
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/webservice.py", line 50, in render_POST
            spiders = get_spider_list(project, version=version)
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/utils.py", line 134, in get_spider_list
            raise RuntimeError(msg.encode('unicode_escape') if six.PY2 else msg)
        builtins.RuntimeError: Traceback (most recent call last):
          File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
            "__main__", mod_spec)
          File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
            exec(code, run_globals)
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 40, in <module>
            main()
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 35, in main
            with project_environment(project):
          File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
            return next(self.gen)
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/runner.py", line 16, in project_environment
            version, eggfile = eggstorage.get(project, eggversion)
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/eggstorage.py", line 28, in get
            version = self.list(project)[-1]
          File "/usr/local/lib/python3.7/dist-packages/scrapyd/eggstorage.py", line 37, in list
            return sorted(versions, key=LooseVersion)
          File "/usr/lib/python3.7/distutils/version.py", line 52, in __lt__
            c = self._cmp(other)
          File "/usr/lib/python3.7/distutils/version.py", line 337, in _cmp
            if self.version < other.version:
        TypeError: '<' not supported between instances of 'str' and 'int'
jpmckinney commented 3 years ago

Closing as duplicate as #212.