Is your feature request related to a problem? Please describe.
Yes, I'd like my crawler restart when it crashed, no matter what the reason is.My Crawler is deployed at an aws machine out of china, it will retrive url from redis which is deployed in the mainland of china. sometimes my crawler will crash because of the network. in this case, I just want to restart my crawler.
Describe the solution you'd like
I want to restart my crawler, just like the --restart unless-stopped in docker run command.
Describe alternatives you've considered
check the crawler every 5min, if it is crashed, restart a new job.
tikazyq
commented
4 years ago
Is your feature request related to a problem? Please describe.
Yes, I'd like my crawler restart when it crashed, no matter what the reason is.My Crawler is deployed at an aws machine out of china, it will retrive url from redis which is deployed in the mainland of china. sometimes my crawler will crash because of the network. in this case, I just want to restart my crawler.
Describe the solution you'd like
I want to restart my crawler, just like the
--restart unless-stoppedindocker runcommand.Describe alternatives you've considered
check the crawler every 5min, if it is crashed, restart a new job.