Link boost and protobuf statically

kareali commented 3 years ago

Motivation

Link Boost and protobuf statically to avoid runtime dependency on either library.
Issue number: #62

Modifications

Change summary

Update the Dockerfile, README.md and CI workflows to build boost with link=static

Testing

docker system prune -a
./docker-build.sh
./docker-run.sh

./localproxy

[2021-08-06T16:56:28.002628]{12}[fatal]   Must specify one and only one of --region/-r or --proxy-endpoint/-e options

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

MiaoZhangAWS commented 3 years ago

Can you explain more in the description on why it needs to be linked statically and what's the issue with dynamically linking? Also why it does not fail before?

MiaoZhangAWS commented 3 years ago

Now our CI action only check compilation. Suggest to add basic sanity check to see if localproxy can starts in CI action. This will help us find issues before checking in the changes.

kareali commented 3 years ago

Now our CI action only check compilation. Suggest to add basic sanity check to see if localproxy can starts in CI action. This will help us find issues before checking in the changes.

@MiaoZhangAWS This is a runtime issue, so it can't be checked in the CI without having some integration test.

kareali commented 3 years ago

Can you explain more in the description on why it needs to be linked statically and what's the issue with dynamically linking? Also why it does not fail before?

@MiaoZhangAWS

static vs dynamic linking
I'm not sure if it was failing before or not, currently building this commit to see.

kareali commented 3 years ago

@MiaoZhangAWS I've considered that it was working before switching to boost 1.76, it's because boost switched the default for the link property from static to dynamic. I will update the PR to do that instead of updating the localproxy cmake command.

MiaoZhangAWS commented 3 years ago

Can you explain more in the description on why it needs to be linked statically and what's the issue with dynamically linking? Also why it does not fail before?

@MiaoZhangAWS
* [static vs dynamic linking](https://cs-fundamentals.com/tech-interview/c/difference-between-static-and-dynamic-linking)

* I'm not sure if it was failing before or not, currently building [this commit](https://github.com/aws-samples/aws-iot-securetunneling-localproxy/commit/f35c8048a4c59923abc11ee863c04186b7345954) to see.

What is not clear to me here is why dynamic linking causes the issue62. For example, why it fails to load the library? I am looking for more analysis here.

kareali commented 3 years ago

Can you explain more in the description on why it needs to be linked statically and what's the issue with dynamically linking? Also why it does not fail before?

@MiaoZhangAWS
* [static vs dynamic linking](https://cs-fundamentals.com/tech-interview/c/difference-between-static-and-dynamic-linking)

* I'm not sure if it was failing before or not, currently building [this commit](https://github.com/aws-samples/aws-iot-securetunneling-localproxy/commit/f35c8048a4c59923abc11ee863c04186b7345954) to see.
What is not clear to me here is why dynamic linking causes the issue62. So I am looking for more analysis here.

Because when you dynamically link a dependency, you need to have that dependency on the OS during the runtime because the binaries only contain a reference to the dynamically linked (shared) library, so if you don't have that dependency installed (which is the case for the docker image because the final image is not the same image where the build happens so it won't have boost on it), the binary will fail during runtime because it can't resolve the reference to the shared library which is exactly what the error in #62 is saying.

./localproxy: error while loading shared libraries: libboost_program_options.so.1.76.0: cannot open shared object file: No such file or directory

Static linking on the other hand would put all the relevant parts of the dependencies in the resulting binary during the linking stage, the resulting binary would contain not just reference to the shared routines of the dependencies, but the actual routines/binary of the used dependencies, so it doesn't need to look in the OS for the those shared libraries. The built binary contains all the necessary parts for it to run, a stand-alone executable. So when you run it on an OS that doesn't have boost installed, it won't matter, because the necessary parts of boost is included in the binary.

MiaoZhangAWS commented 3 years ago

Can you explain more in the description on why it needs to be linked statically and what's the issue with dynamically linking? Also why it does not fail before?

@MiaoZhangAWS
* [static vs dynamic linking](https://cs-fundamentals.com/tech-interview/c/difference-between-static-and-dynamic-linking)

* I'm not sure if it was failing before or not, currently building [this commit](https://github.com/aws-samples/aws-iot-securetunneling-localproxy/commit/f35c8048a4c59923abc11ee863c04186b7345954) to see.
What is not clear to me here is why dynamic linking causes the issue62. So I am looking for more analysis here.
Because when you dynamically link a dependency, you need to have that dependency on the OS during the runtime because the binaries only contain a reference to the dynamically linked (shared) library, so if you don't have that dependency installed (which is the case for the docker image because the final image is not the same image where the build happens so it won't have boost on it), the binary will fail during runtime because it can't resolve the reference to the shared library which is exactly what the error in #62 is saying.
./localproxy: error while loading shared libraries: libboost_program_options.so.1.76.0: cannot open shared object file: No such file or directory
Static linking on the other hand would put all the relevant parts of the dependencies in the resulting binary during the linking stage, the resulting binary would contain not just reference to the shared routines of the dependencies, but the actual routines/binary of the used dependencies, so it doesn't need to look in the OS for the those shared libraries. The built binary contains all the necessary parts for it to run, a stand-alone executable. So when you run it on an OS that doesn't have boost installed, it won't matter, because the necessary parts of boost is included in the binary.

Thanks for your clarification. It makes sense!

aws-samples / aws-iot-securetunneling-localproxy