ops issues

[Evanfeenstra]

crashing ec2

• limit memory per container, and total docker limit?
• rate limiting in traefik
• logs outside sometimes - cloudwatch. also make sure to have good logs
• log rotation if its local

ip addresses changing

• static IPs on lightning nodes ($3/month)
• could still have a load balancer (for domains) Forward to traefik

better logs in swarm UI

[Evanfeenstra]

superadmin

• creating swarms
• restarting EC2
• update route53 stuff

[tomsmith8]

@Evanfeenstra could you prioritise setting docker and container limits.

@kevkevinpal could you prioritise migrating the btc graph, updating the github actions pipeline and deprecating the non swarm ec2 instances

Next up then would be setting up cloud watch?

[Evanfeenstra]

just merged a per container memory limit, set it once and it applies to every container

https://github.com/stakwork/sphinx-swarm/commit/84ab2259b96e11dfc8e899639b518e53d012489c

Its global_mem_limit in the yaml config file, its a number in bytes

[Evanfeenstra]

@tobi-bams here's a new SetGlobalMemLimit cmd, maybe u can add a frontend for it? https://github.com/stakwork/sphinx-swarm/blob/master/src/cmd.rs#L152

[tobi-bams]

@tobi-bams here's a new SetGlobalMemLimit cmd, maybe u can add a frontend for it? https://github.com/stakwork/sphinx-swarm/blob/master/src/cmd.rs#L152

Yea, sure I can.

[Evanfeenstra]

log rotation: https://github.com/stakwork/sphinx-swarm/releases/tag/v0.4.98

[tomsmith8]

Update all swarms to m5.large or higher.

Do not use t groups due to CPU credits and spikes causes machines to become unavailable.

[tomsmith8]

@Evanfeenstra any updates on keeping logs?

• not deleting and keeping locally
• future -> stream logs to something like cloudwatch