Dynamically loading a user provided module does not work

[swendlandt]

What version of Cloud Foundry and CF CLI are you using? (i.e. What is the output of running cf curl /v2/info && cf version? "api_version":"2.203.0" cf version 8.5.0+73aa161.2022-09-12

What version of the buildpack you are using? 1.2.4

If you were attempting to accomplish a task, what was it you were attempting to do? Dynamically loading a user provided module as described in the docs: https://docs.cloudfoundry.org/buildpacks/nginx/index.html I'm testing with a minimal setup: manifest.yml

---
applications:
- name: myapp
  memory: 32M
  instances: 2
  buildpacks:
    - https://github.com/cloudfoundry/nginx-buildpack.git#v1.2.4

nginx.conf:

{{module "ngx_http_geoip2_module"}}

events {}

http {
  server {
    listen {{port}};
  }
}

The file "ngx_http_geoip2_module.so" is located in the "modules" folder which is located in the root folder of my app on the same level as nginx.conf and manifest.yml.

When I push the app it fails with BuildpackCompileFailed - App staging failed in the buildpack compile phase. In the logs of the app I can see this:

2023-07-03T15:13:28.68+0200 [STG/0] ERR 2023/07/03 13:13:28 [emerg] 903#0: dlopen() "/tmp/app/modules/ngx_http_geoip2_module.so" failed (/tmp/app/modules/ngx_http_geoip2_module.so: cannot open shared object file: No such file or directory) in /tmp/conf290931122/nginx.conf:1
   2023-07-03T15:13:28.68+0200 [STG/0] ERR nginx: configuration file /tmp/conf290931122/nginx.conf test failed

What did you expect to happen? Expected it to successfully load the module or at least that it can find the shared object file.

What was the actual behavior?

Please confirm where necessary:

• [x] I have included a log output
• [x] My log includes an error message
• [x] I have included steps for reproduction

[arjun024]

Not sure exactly why that happens. Would you be able to provide a full sample app?

One of my suspicions is that the error message could be misleading. Is the module statically built? Does it have any dynamic dependencies?

[swendlandt]

I tried with different modules and always get the same error. ngx_http_geoip_module, ngx_http_geoip2_module and this hello world module: https://github.com/perusio/nginx-hello-world-module

Here is a very minimal example of the app: https://github.com/swendlandt/nginx-buildpack-example/tree/main

[arjun024]

@swendlandt I followed the instructions on the nginx blog (https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus/) to build the hello-world dynamic module. I built it against nginx 1.25.1 (you have to use the exact same version for dynamic modules). I copied that module into my app here https://github.com/arjun024/nginx-dynamic-module-app and cf push testapp -b https://github.com/cloudfoundry/nginx-buildpack#v1.2.5. Everything works fine!

My guess is that you didn't build against the specific version and this is a red herring error from NGINX. I'm going to close the issue - feel free to reopen it. If you figure out the issue, let us know

[swendlandt]

@arjun024 Thanks for your support. With your example app it worked directly for me. I also got it to work now. The error was indeed a red herring, however in my case the problem was not the NGINX version, but rather the platform on which the .so-file was built. I tried to built the module inside a Docker container running Ubuntu before, but the problem was my host machine is an M1-Macbook with ARM64-architecture. So what I did to build the module was to spin up a container emulating the AMD64 platform with docker run -it --platform linux/amd64 ubuntu /bin/bash. Then I built the module as described here (https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus/) in the container. With the resulting file it finally worked.