search

freezingsaddles / freezing-compose

The Docker Compose structure for running the application suite.
4 stars 2 forks source link

Send logs to Cloudwatch logs #18

Closed obscurerichard closed 2 years ago

obscurerichard commented 4 years ago

See https://github.com/mdsol/logspout-cloudwatch

Right now logspout is useless because the subscription it tries to send to is lapsed.

We could also remove logspout entirely and rely on Docker's ability to send logs directly to Cloudwatch, see:

obscurerichard commented 2 years ago

In order to do this, I think we are going to have to move the Freezing Saddles server out of Lightsail to EC2. Which also means we are going to have to move the PostgreSQL server from Lightsail to RDS. Ugh.

So the first thing to do is to move the PostgreSQL server, and then move the Lightsail server.

AWS has an article on migrating RDS here: https://aws.amazon.com/blogs/compute/amazon-lightsail-database-tips-and-tricks/

I already did the first step, enabling VPC peering for LightSail; and boom just like that a VPC peering connection showed up in the default VPC in us-east-1 where these resources live.

I did back up and restores for the MySQL databases to the new RDS server. I had to do some surgery on the freezing dump file to remove some annotations in the view definition about what the definer user was to get freezing to load cleanly. Then I reconfigured the server to use the new Aurora database which worked fine.

Then I stopped all the services and created a snapshot of the Lightsail instance and then exported that snapshot to EC2.

obscurerichard commented 2 years ago

I set up new private subnets in all 6 availability zones in us-east-1, then added a new RDS DB subnet group that includes all of them called unmanaged-fs-db-sg that has all 6 subnets in it. I then created an AWS RDS Aurora MySQL 5.6 compatible database with t3.small instances called unmanged-fs-db-sg that has an AWS-created security group called unmanaged-fs-rds-vpc-sg.

I created another VPC security group called unmanaged-fs-ec2-sg and had it allow SSH and HTTP and HTTPS from the world, and then modified the unmanaged-fs-rds-vpc-sg to allow access both from the EC2 SG and from 172.26.0.0/16, the Lightsail network. This should allow us to connect to the new RDS server from the Lightsail instance.

obscurerichard commented 2 years ago

I created a new IAM policy and role to allow CloudWatch logs write access, and allocated a new elastic IP address. I fixed up Route53 to point to the new IP address 54.84.77.223, then I created the EC2 instance, put it a public subnet in us-east-1a, and associated the elastic IP address. It started clean and allowed me to login, and then I started the docker-compose services again (except for freezing-sync which I immediately stopped).

obscurerichard commented 2 years ago

Now we are ready to send logs to CloudWatch - we just have to configure either logspout or Docker itself to do it.

obscurerichard commented 2 years ago

I configured Logspout to send logs to CloudWatch logs.