iscsiadm links to many libraries on 2697.0.0

[iaguis]

Description

On 2697.0.0, the iscsiadm binary links to many more libraries than in previous version. This causes problems because in Lokomotive we mount it in the kubelet container, which misses some of the libraries.

Impact

Kubernetes workloads depending on the iscsiadm binary fail. This would probably break any distro that runs the kubelet in a container.

Environment and steps to reproduce

1. Set-up: Flatcar 2697.0.0 on Packet
2. Task: Run ldd $(which iscsiadm)

Expected behavior

I expect iscsiadm to be linked to the same libraries as previous versions.

Additional information

• Flatcar 2697.0.0

  $ ldd $(which iscsiadm)
  linux-vdso.so.1 (0x00007ffceacf0000)
  libisns.so.0 => /lib64/libisns.so.0 (0x00007f2d8fdc2000)
  libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f2d8fb14000)
  libmount.so.1 => /lib64/libmount.so.1 (0x00007f2d8fab6000)
  libkmod.so.2 => /lib64/libkmod.so.2 (0x00007f2d8fa9e000)
  libopeniscsiusr.so.0.2.0 => /lib64/libopeniscsiusr.so.0.2.0 (0x00007f2d8fa7e000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f2d8f8a6000)
  libz.so.1 => /lib64/libz.so.1 (0x00007f2d8f88a000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f2d8f884000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2d8f860000)
  libblkid.so.1 => /lib64/libblkid.so.1 (0x00007f2d8f80b000)
  librt.so.1 => /lib64/librt.so.1 (0x00007f2d8f801000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f2d8fec4000)
  liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f2d8f7d8000)
  libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f2d8f7cd000)

• Flatcar 2605.8.0

  $ ldd $(which iscsiadm)
  linux-vdso.so.1 (0x00007ffdfcf4d000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f8aca2a4000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f8aca550000)

This was probably introduced in https://github.com/kinvolk/coreos-overlay/pull/682.

I could work it around by mounting the missing libraries into the kubelet container, but I was wondering if this could be solved on Flatcar.

[pothos]

So you would run the binary with the libraries you have in a container? That's not what is guaranteed to work as dependencies will change on updates. It was not a static binary before. Can you copy/mount the libraries alongside (ldd $(which iscsiadm) | grep -o /lib'[^ ]*' | sort -u)?

[t-lo]

As raised by @pothos I'm not sure how this is a bug. We updated the version of open-iscsi to latest upstream to address security issues with the outdated version, and to ship bug fixes. The tools are not shipped as static binaries, so run-time compatibility to 3rd party containers cannot be guaranteed.

In fact, since C only supports compile-time type safety but no run-time type safety, this might ave introduced silent breakage with previous versions already when using application binaries not built against the libraries shipped. If e.g. a library function signature changes - for instance, if there's a type change in a function parameter or if a function changes the number of arguments, things would continue to run, but application and library would make different assumptions on said function calls. This kind of operation is not safe.

[pothos]

To add, there comes a glibc update to Stable. I think it would work in one direction; using an old binary with a newer library as long at the SONAME (library major version) is the same but maybe not in the reverse direction when new symbols are used.

[pothos]

@iaguis I proposed another approach here: https://github.com/kinvolk/Flatcar/issues/293#issuecomment-742673711