scrapinghub / scrapyrt

HTTP API for Scrapy spiders
BSD 3-Clause "New" or "Revised" License
824 stars 161 forks source link

How to host my scrapy project which uses scrapyRT ? #160

Closed dev-divyansh closed 4 months ago

dev-divyansh commented 4 months ago

When using scrapyRT I can dynamically enter the url and that would give me the json response but this is on the local machine. I want to host this and whenever I make a request with some url it should give me the json response.

Simply I am asking how I can host my spider and dynamically pass the url and get the response.

pawelmhm commented 4 months ago

There is a dockerfile here https://github.com/scrapinghub/scrapyrt/blob/master/Dockerfile which you can use to run it in your cloud. How to do this exactly will depend on your requirements and cloud provider you will use. We don't really provide info on this, because this is very user-specific, frequently changing and requires lots of time and attention and sysadmin expertise