scrapy / scrapyd

A service daemon to run Scrapy spiders
https://scrapyd.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
2.93k stars 571 forks source link

facing unexpected builtins.TypeError error when running >Scrapyd command . #456

Closed usmanabbasi360 closed 1 year ago

usmanabbasi360 commented 1 year ago

using WSL - Linux I am trying to deploy my scrapy spider locally. I have installed all the required libraries but when I run command $ scrapyd it throughs an error the error is not related to my project it is related to scrapyd library as we can see the error path try to search for the error but find solution nowhere .

          File "/mnt/d/webscrapy/.webscrapingenv2/lib/python3.8/site-packages/scrapyd/website.py", line 212, in prep_tab_finished
            return '\n'.join(
        builtins.TypeError: 'NoneType' object is not iterable
jpmckinney commented 1 year ago

What is the content of your scrapyd.conf file?

usmanabbasi360 commented 1 year ago

@jpmckinney here it is [settings] default = hms.settings

[deploy:local] url = http://127.0.0.1:6800/ project = hms

jpmckinney commented 1 year ago

That's your scrapy.cfg file. I need scrapyd.conf. Looks like:

[scrapyd]
eggs_dir    = eggs
logs_dir    = logs
items_dir   =
jobs_to_keep = 5
dbs_dir     = dbs
max_proc    = 0
max_proc_per_cpu = 4
finished_to_keep = 100
poll_interval = 5.0
bind_address = 127.0.0.1
http_port   = 6800
username    =
password    =
debug       = off
runner      = scrapyd.runner
jobstorage  = scrapyd.jobstorage.MemoryJobStorage
application = scrapyd.app.application
launcher    = scrapyd.launcher.Launcher
webroot     = scrapyd.website.Root
eggstorage  = scrapyd.eggstorage.FilesystemEggStorage

[services]
schedule.json     = scrapyd.webservice.Schedule
cancel.json       = scrapyd.webservice.Cancel
addversion.json   = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json  = scrapyd.webservice.ListSpiders
delproject.json   = scrapyd.webservice.DeleteProject
delversion.json   = scrapyd.webservice.DeleteVersion
listjobs.json     = scrapyd.webservice.ListJobs
daemonstatus.json = scrapyd.webservice.DaemonStatus
usmanabbasi360 commented 1 year ago

@jpmckinney I have no file with this name..

jpmckinney commented 1 year ago

@usmanabbasi360 Please create one like at https://scrapyd.readthedocs.io/en/stable/config.html and restart the scrapyd service. It will make it easier to debug your issue, which seems impossible right now.

jpmckinney commented 1 year ago

Closing as no response.