aws / aws-sdk-rails

Official repository for the aws-sdk-rails gem, which integrates the AWS SDK for Ruby with Ruby on Rails.
Other
596 stars 62 forks source link

Check existence of dockerenv to verify in docker container, to correct issue in AL2023 #155

Closed ssunday closed 2 weeks ago

ssunday commented 2 weeks ago

Issue #, if available:

https://github.com/aws/aws-sdk-rails/issues/154

Description of changes:

Solution taken from https://github.com/active-elastic-job/active-elastic-job/pull/154 โ€” trying to create a path to deprecate that gem and recommend this gem, and this is an issue that prevents a clear migration ๐Ÿ˜ .

Not sure if we need to ask that PR author for permission, it was merged into an MIT repo.

mullermp commented 2 weeks ago

Can you give more details about your docker version and environment? Last I looked into this, this was a dangerous check to have.

ssunday commented 2 weeks ago

@mullermp I'm not the one that figured out this fixโ€”the issue is on latest Elastic Beanstalk Docker AL2023 environments. The original PR describes the issue.

ssunday commented 2 weeks ago

It looks like we could also use an environment variable to bypass all of this, if the dockerenv is an uncomfortable change. Comments on https://stackoverflow.com/questions/23513045/how-to-check-if-a-process-is-running-inside-docker-container suggest /proc/self/mountinfo might work somehow, but I don't know. But the 0::/ seems like a thing that happens on certain docker host machines...

ssunday commented 2 weeks ago

Oh, duh, it does check /proc/self/mountinfo which clearly doesn't work in this case then lol

mullermp commented 2 weeks ago

Are you able to get the docker image version on those elastic beanstalk containers? I'm trying to setup a reproduction environment now. When I use docker locally and pull the latest amazonlinux:2023 image, /proc/self/mountinfo returns paths with docker/containers as expected.

ssunday commented 2 weeks ago

@mullermp I guess I'm confused with your askโ€”the issue is the host being AL2023, not the docker image being AL2023.

ssunday commented 2 weeks ago

@mullermp Like the issue surfaces when using AL2023 rather than AL2 and running docker container on it.

mullermp commented 2 weeks ago

I believe it. I'm unable to reproduce though. I am on an AL2023 host provisioned by elastic beanstalk:

[ec2-user@ip-172-31-10-22 ~]$ uname -a
Linux ip-172-31-10-22.us-west-2.compute.internal 6.1.111-120.187.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Sep 20 14:52:08 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

and from a docker shell (I am just using the bash one):

sudo docker pull bash
sudo docker run -it bash /bin/sh

I try to check cgroup and mountinfo. /proc/1/cgroup comes back as 0::1/, but /proc/self/mountinfo does have paths with docker/containers in them.

I don't use docker often so I could be doing something wrong. Do you have a specific image that you are running from AL2023?

I think worst case we can add an ENV variable as an override for using docker here.

ssunday commented 2 weeks ago

@mullermp I do see docker/containers in one project's /proc/self/mountinfo, yet it didn't work when I actually used this gem. Is it possible the regex / how the code is working is busted? I'm guessing usage of this gem in such docker environments is low (hence why this wasn't raised until now ๐Ÿ˜) so who knows on real world verification. Which is weird as it IS tested in a way that looks akin to reality...

BUT. Maybe the issue is with the IP checking, actually? The other gem (which is known to work) does it differently https://github.com/active-elastic-job/active-elastic-job/blob/master/lib/active_elastic_job/rack/sqs_message_consumer.rb#L139

I quickly assumed it was the lack of dockerenv check, but maybe I was wrong. I didn't actually test this on any project, I was so certain ๐Ÿ˜‚

ssunday commented 2 weeks ago

@mullermp The other gem's IP check looks superior (?), fwiw. I can replace this gem's IP code with that one and actually test it in reality (gasp).

ssunday commented 2 weeks ago

I'm leaning towards IP being the issue as I ran the mountinfo code in a rails console and it worked. TBD on the IP changes! I'll close and re-raise if I figure out/verify it works.

ssunday commented 2 weeks ago

Yes, it seems to be an IP issue