Example workloads failing on CentOS after "Decouple all examples.." commit

jinengandhi-intel commented 3 years ago

Description of the problem

With this commit the method to compute the ARCH_LIBDIR has changed and this change is causing most Example workloads to fail. https://github.com/oscarlab/graphene/pull/2582/files

Error message from most workloads:

[intel1@localhost python]$ graphene-direct ./python scripts/helloworld.py
error: Using insecure argv source. Graphene will continue application execution, but this configuration must not be used in production!
[P1:T1:] error: Mounting file:/lib/x86_64-redhat-linux on /lib/x86_64-redhat-linux (type=chroot) failed (2)
[P1:T1:] error: Error during shim_init() in init_mount (-2)

Essentially the gcc -dumpmachine is resolving to x86_64-redhat-linux whereas this path doesn't exist on the system. As per our CentOS machine all the contents that would generally be present in /lib/{ARCH_LIBDIR} folder is instead present in /usr/lib64. Upon adding this to the fs.mount and the LD_LIBRARY_PATH the workload seems to PASS.

Steps to reproduce

Run python, r, bash etc. workloads on CentOS system.

Expected results

Actual results

dimakuv commented 3 years ago

Yes, this is expected. I put this comment in the Redis example: https://github.com/oscarlab/graphene/blob/master/Examples/redis/Makefile#L22-L25

Basically, try building all examples like this: ARCH_LIBDIR=/lib64 make. This way you don't need to change fs.mount in the manifest templates of each example.

jinengandhi-intel commented 3 years ago

Yes, we have tried it manually and it works, but shouldn't there be a generic way of generating the manifest files for the examples? or we call them out as not-applicable for CentOS? In it's current form they are failing and there are customers who are trying Graphene with CentOS who will face the error and report back.

dimakuv commented 3 years ago

When merging that "Decouple all examples..." commit, we agreed that we currently only say about Ubuntu 18.04/20.04 support out of the box. The other distros currently require tweaks.

It's unclear how to make all our Makefiles choose between Ubuntu and CentOS peculiarities in a reasonable way. I'm more than open to suggestions (but not something like if os == centos then arch_libdir="/lib64", that's just hard-coding that will fail on yet another distro).

jinengandhi-intel commented 3 years ago

I was going through Quick Start Guide and there we have a section called "Running Sample Applications" where we ask users to try the examples, can we add a note here that the current examples only supports Ubuntu distro? https://graphene.readthedocs.io/en/latest/quickstart.html#running-sample-applications

dimakuv commented 3 years ago

@jinengandhi-intel Good suggestion, let me create a quick PR with this note.

jinengandhi-intel commented 3 years ago

Issue is verified.

gramineproject / graphene