feat: dockerize scraper

icssc / peterportal-api-next

API that provides easy access to public data from UC Irvine. Developed for Anteaters, by Anteaters.

https://docs.icssc.club/anteaterapi

MIT License

6 stars 0 forks source link

feat: dockerize scraper #26

Closed ecxyzzy closed 1 year ago

ecxyzzy commented 1 year ago

TODO:

[x] Investigate whether it is possible to SSH into the EC2 instance managed by ECS

ecxyzzy commented 1 year ago

For the record, it is not possible [to enable external SSH access] without significant manual screwing-around (outside of CDK). Since the point of using CDK is to ensure that our infrastructure is replicable, there's not really much point in doing so.

We can however get the container to output logs to CloudWatch, so that's a win in my books.

ecxyzzy commented 1 year ago

Scraper seems to not be reading the previous exit time properly, since it's scraping every term on start. Will do some more testing.

ecxyzzy commented 1 year ago

Will hold off on merging this until we have a successful deploy, because the scraper's CloudFormation stack is currently stuck in limbo.