Are certain commands locked down?

[philiiiiiipp]

Hey,

I am trying to use jq in a shell script, but got command not found. I then tried to install it within my Dockerfile:

RUN yum install -y epel-release
RUN yum install -y jq
RUN jq --version

But it first says that its already installed and then does not find it:

Step 4/10 : RUN yum install -y epel-release
 ---> Running in 97f473ea4dbe
Loaded plugins: ovl
Package epel-release-6-8.9.amzn1.noarch already installed and latest version
Nothing to do
Removing intermediate container 97f473ea4dbe
 ---> d6a4a94912df
Step 5/10 : RUN yum install -y jq
 ---> Running in 7465dd16597a
Loaded plugins: ovl
Package jq-1.5-1.2.amzn1.x86_64 already installed and latest version
Nothing to do
Removing intermediate container 7465dd16597a
 ---> 484fd31be453
Step 6/10 : RUN jq --version
 ---> Running in 60bce46c23ef
/bin/sh: jq: command not found
The command '/bin/sh -c jq --version' returned a non-zero code: 127

Are those commands somehow locked down?

[mhart]

Which image is this?

[philiiiiiipp]

It's the lambci/lambda:build-nodejs8.10

[mhart]

Interesting – it's actually installed on Lambda itself – but it looks like they've removed the binaries:

docker run --rm --entrypoint sh lambci/lambda:nodejs8.10 -c 'yum list installed | grep jq && find / -name jq'
jq.x86_64                     1.5-1.2.amzn1                 installed           
jq-libs.x86_64                1.5-1.2.amzn1                 installed           
find: `/root': Permission denied
find: `/var/lib/yum/history/2019-01-31/1': Permission denied
find: `/proc/tty/driver': Permission denied

So it will need to be reinstalled. I'll do this in the build images, but you can also do it yourself:

docker run --rm lambci/lambda:build-nodejs8.10 sh -c 'yum reinstall -y jq && find / -name jq'
Loaded plugins: ovl
Resolving Dependencies
--> Running transaction check
---> Package jq.x86_64 0:1.5-1.2.amzn1 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package     Arch            Version                   Repository          Size
================================================================================
Reinstalling:
 jq          x86_64          1.5-1.2.amzn1             amzn-main           44 k

Transaction Summary
================================================================================
Reinstall  1 Package

Total download size: 44 k
Installed size: 57 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : jq-1.5-1.2.amzn1.x86_64                                      1/1 
  Verifying  : jq-1.5-1.2.amzn1.x86_64                                      1/1 

Installed:
  jq.x86_64 0:1.5-1.2.amzn1                                                     

Complete!
/usr/share/doc/jq
/usr/bin/jq

So just use the following, instead of the install line:

RUN yum reinstall -y jq

[philiiiiiipp]

Ok that makes sense, it seems as if they are removing more though. Reinstalling works fine, but if I am trying to run I get:

jq: error while loading shared libraries: libjq.so.1: cannot open shared object file: No such file or directory

Full output:

Step 4/5 : RUN yum reinstall -y jq
 ---> Running in cd330921c239
Loaded plugins: ovl
Resolving Dependencies
--> Running transaction check
---> Package jq.x86_64 0:1.5-1.2.amzn1 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package     Arch            Version                   Repository          Size
================================================================================
Reinstalling:
 jq          x86_64          1.5-1.2.amzn1             amzn-main           44 k

Transaction Summary
================================================================================
Reinstall  1 Package

Total download size: 44 k
Installed size: 57 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : jq-1.5-1.2.amzn1.x86_64                                      1/1 
  Verifying  : jq-1.5-1.2.amzn1.x86_64                                      1/1 

Installed:
  jq.x86_64 0:1.5-1.2.amzn1                                                     

Complete!
Removing intermediate container cd330921c239
 ---> 587f4caa061d
Step 5/5 : RUN jq --version
 ---> Running in a6768b3d9c50
jq: error while loading shared libraries: libjq.so.1: cannot open shared object file: No such file or directory
The command '/bin/sh -c jq --version' returned a non-zero code: 127

[mhart]

yum reinstall -y jq-libs jq

[philiiiiiipp]

I got it running by also adding oniguruma

yum reinstall -y jq jq-libs oniguruma

Thanks!

Should I close this issue or are you going to do it when an update to the images is available?

[mhart]

I'll update when I figure out the best way to deal with packages that are seemingly installed... but actually aren't

Thanks for the heads up!

[philiiiiiipp]

In my investigation I stumbled upon compgen -c which might work in combination with a yum list installed.

Hope this helps, thanks for the awesome work on this package!

[kadrach]

We just noticed this issue on current lambci/lambda:build-python3.6as well.

[mhart]

@kadrach it'll be the same on all build images. I'm still waiting to hear from AWS on the best solution for these sorts of issues – I can fix it just for jq, but I'd prefer to fix it for all packages that report as being installed, but actually aren't.

[mhart]

This is fixed (or at least should be) by 23357e247b94210b58f5c4af2d98f54338aa4d28

All build images have been updated