search

shaye059 / Machine-Learning-Stock-Prediction

Using NLP Pipeline and Machine learning to anticipate the effect of news stories on stock prices.
MIT License
0 stars 1 forks source link

Create an ETL pipeline #2

Open shaye059 opened 3 years ago

shaye059 commented 3 years ago

Create one complete pipeline to run the scraping and preprocessing - getting the news articles, calculating the before and after days, getting the stock prices - and set it up as a scheduled cron task on ECS.

The entire pipeline will have to be dockerized.

Setting up scheduled tasks on ECS: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduled_tasks.html

shaye059 commented 3 years ago

Preferably this would be fault tolerant and modular so that it can make use of EC2 spot instances https://aws.amazon.com/ec2/spot/?cards.sort-by=item.additionalFields.startDateTime&cards.sort-order=asc