search

claffin / cloudproxy

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.
https://cloudproxy.io/
MIT License
1.4k stars 79 forks source link

Cancel spot requests when associated instances are terminated #48

Closed henryzxu closed 3 years ago

henryzxu commented 3 years ago

When deleting proxies filled by one-time spot requests, the instances are terminated, but the spot request itself is not cancelled, leaving the door open to being filled again in the future when not associated with cloudproxy. This PR deletes the spot request if it exists in the delete_proxy() function. Related to #42, but unclear if applicable to persistent spot requests.

henryzxu commented 3 years ago

Looking into this a little further, per https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html, it looks like one-time requests are closed if the instance is terminated, and this pr proactively cancels the request instead of waiting for ec2 to do it eventually.

Additional testing seems to indicate we can also cancel persistent spot requests and thus prevent future undesired spin ups of associated instances per #42. I've updated the condition to be applicable to both spot requests accordingly.

claffin commented 3 years ago

Excellent PR, thank you for your contribution.

I've tested this now, it does what's described and fixes issue #42. I will merge.