scrapinghub / scrapyrt

HTTP API for Scrapy spiders
BSD 3-Clause "New" or "Revised" License
824 stars 161 forks source link

How do you run scrapyrt with forever.js or pm2? #124

Closed doverradio closed 3 years ago

doverradio commented 3 years ago

Hi,

I moved a project to a live server and now I need something like pm2 or forever.js to run scrapyrt.

However, neither command seemed to work...

forever.js

I ran this command which failed (and I was in an active virtual environment):

# forever start scrapyrt -p 5003
warn:    --minUptime not set. Defaulting to: 1000ms
warn:    --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info:    Forever processing file: scrapyrt
error:   Cannot start forever
error:   script /root/my-scrapy-project/scrapyrt does not exist.

pm2

# pm2 start scrapyrt -p 5003
# pm2 list
┌─────┬─────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name        │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼─────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0   │ scrapyrt    │ default     │ N/A     │ fork    │ 0        │ 0      │ 15   │ errored   │ 0%       │ 0b       │ root     │ disabled │
└─────┴─────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘

How do you set scrapyrt to permanently be turned on and automatically restarted like with pm2 or forever?

doverradio commented 3 years ago

Any way to do this anyone? How do you keep scrapyrt permanently up?

doverradio commented 3 years ago

I found that if the server IP address is 65.124.80.15 and I want to run scrapyrt on port 5003 off it, I can use the following commands to get pm2 running scrapyrt...

cd <into_scrapy_projectfile_containing_scrapy.cfg_file>
pm2 start "scrapyrt -p 5003 -i 65.124.80.15"