search

josephmachado / beginner_de_project

Beginner data engineering project - batch edition
https://www.startdataengineering.com/post/data-engineering-project-for-beginners-batch-edition/
MIT License
464 stars 136 forks source link

aws redshift not created #14

Closed shivnarayanan closed 2 years ago

shivnarayanan commented 2 years ago
image
josephmachado commented 2 years ago

@shivnarayanan It looks like redshift cluster is created, but the problem is with Airflow webserver docker container. Can you

  1. Check if Redshift cluster is created in AWS UI?
  2. Check if the webserver and scheduler containers are running using docker ls
  3. Make sure your docker setting has at least 4GB memory.

LMK

shivnarayanan commented 2 years ago
image

@josephmachado Hello! I just realised I receive this error halfway through. Have I done something wrong?

josephmachado commented 2 years ago

@shivnarayanan Could you please check if your docker setting has at least 4GB memory ?

kokoWoo commented 2 years ago

You need to check airflow-webserver1 of docker container and airflow-webserver1 of setup_infra.sh. setup_infra.sh written beginner_de_project_airflow_webserver_1.

VentureFurther commented 2 years ago
image

@shivnarayanan The container name is incorrect in the old script setup_infra.sh It should be beginner_de_project-airflow-webserver-1, but the one in the script is beginner_de_project_airflow-webserver_1. Difference being that "_" should be "-". I had the same errors and fixed it by doing this.

josephmachado commented 2 years ago

@shivnarayanan I put up a fix for this, can you pull master branch and try again?