search

scrapinghub / python-scrapinghub

A client interface for Scrapinghub's API
https://python-scrapinghub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
202 stars 63 forks source link

How to get job_arguments? #161

Closed bulatbulat48 closed 3 years ago

bulatbulat48 commented 3 years ago

We can run the jobs with job_args. 

project.jobs.run('spider1', job_args={'arg1': 'val1'})

but we can't get arguments from finished jobs:

project.jobs.list(count=1, spider='spider')
bulatbulat48 commented 3 years ago

Looks like Scrapy Cloud doesn't have job arguments method in the API: https://docs.zyte.com/scrapy-cloud/jobq.html#jobq-project-id-list

hermit-crab commented 3 years ago

Good day @bulatbulat48. I believe you can get the job arguments via

project.jobs.list(count=1, spider='spider', meta=['spider_args'])

Other keys can be found at https://docs.zyte.com/scrapy-cloud/jobmeta.html. Although not sure where the meta API parameter itself is documented.

bulatbulat48 commented 3 years ago

@hermit-crab it works! Thank you!