Start Coding Process - Githubissues

[x] write a script for getting api live data form one sources and write them in a file with python
[x] move get live data script to docker
[x] change file to redis and
[x] determine first stage log for each part
[x] add airflow docker
[x] add postgres and redis database
[x] create a script to make a simple process to redis data and transfer it to postgres
[x] create a docker image and a DAG for process script
[x] create a database manager to automate postgres table creation
[ ] create a script for historical data from at least two source
[ ] add a DAG for historical data script
[ ] add a process to create time frame data from tick data
[ ] enhance process and add feature engineering
[ ] add ICT mathematical tools
[ ] enhance code structure
[ ] add unit test
[x] centralize all logs using ELK
[ ] consider using CI/CD
[ ] automate the process
[ ] add api interface to manage the automated process
[ ] consider using kubernetes for orchestration
[ ] config airflow for kubernetes
[ ] consider using Prometheus and grafana
[ ] add chart and visualization for cleaned data

wj-turner / OmegaFin