search

sclorg / container-common-scripts

Apache License 2.0
20 stars 45 forks source link

Add support docker_args for `ct_build_image_and_parse_id` #363

Closed phracek closed 6 months ago

phracek commented 6 months ago

This commit was tested on two containers. nginx-container and s2i-nodejs-container.

nginx-container uses https://github.com/sclorg/nginx-container/blob/2ba264983b1236c6299eb83c62e4db9f40959f36/test/run#L418 s2i-nodejs-container pull request is here: https://github.com/sclorg/s2i-nodejs-container/pull/421

In whole SCLORG organization https://github.com/search?q=org%3Asclorg++ct_test_app_dockerfile&type=code

It is not used port as a parameter at all, therefore removing it from function as a parameter makes sense.

nginx-container test status:

Running test run_dockerfiles_test (starting at 2024-03-12 04:14:16-04:00) ...
-----------------------------------------------
Using this Dockerfile:
FROM ubi8/nginx-122:1

ENV NGINX_VERSION=1.22

ADD nginx-container/examples/1.22/test-app/nginx.conf "${NGINX_CONF_PATH}"
ADD nginx-container/examples/1.22/test-app/nginx-default-cfg/*.conf "${NGINX_DEFAULT_CONF_PATH}"
ADD nginx-container/examples/1.22/test-app/nginx-cfg/*.conf "${NGINX_CONFIGURATION_PATH}"
ADD nginx-container/examples/1.22/test-app/*.html ./

CMD nginx -g "daemon off;"
Cloning into 'nginx-container'...
remote: Enumerating objects: 2913, done.
remote: Counting objects: 100% (929/929), done.
remote: Compressing objects: 100% (210/210), done.
remote: Total 2913 (delta 780), reused 782 (delta 716), pack-reused 1984
Receiving objects: 100% (2913/2913), 526.15 KiB | 13.15 MiB/s, done.
Resolving deltas: 100% (1760/1760), done.
Building 'myapp' image using docker build
STEP 1/7: FROM ubi8/nginx-122:1
STEP 2/7: ENV NGINX_VERSION=1.22
--> e56b32fe6508
STEP 3/7: ADD nginx-container/examples/1.22/test-app/nginx.conf "${NGINX_CONF_PATH}"
--> 73028c318591
STEP 4/7: ADD nginx-container/examples/1.22/test-app/nginx-default-cfg/*.conf "${NGINX_DEFAULT_CONF_PATH}"
--> d9c1aaa92cfa
STEP 5/7: ADD nginx-container/examples/1.22/test-app/nginx-cfg/*.conf "${NGINX_CONFIGURATION_PATH}"
--> bf1bdc45252b
STEP 6/7: ADD nginx-container/examples/1.22/test-app/*.html ./
--> d1dcad6766c7
STEP 7/7: CMD nginx -g "daemon off;"
COMMIT myapp
--> 6fb02d0904c2
Successfully tagged localhost/myapp:latest
6fb02d0904c2d46c02e57471abf4c48861a3d6f53efd6e9cd6d005b1076009d0
b8c9e2de19e5e4b6f8631520d4be7d87f91635a9965d37cda55286a028c26b1b
Waiting for myapp to start
  Testing the HTTP(S) response for <http://10.88.0.47:8080>
Trying to connect ... 1
b8c9e2de19e5e4b6f8631520d4be7d87f91635a9965d37cda55286a028c26b1b

s2i-nodejs-container test status:

Running test test_check_build_using_dockerfile (starting at 2024-03-12 04:25:09-04:00) ...
-----------------------------------------------

[INFO] Check building using a Dockerfile...

Using this Dockerfile:
FROM ubi8/nodejs-20:1

ADD app-src .

USER 0
RUN fix-permissions ./
USER 1001

RUN npm install

CMD npm run -d start
Cloning into 'app-src'...
remote: Enumerating objects: 782, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (69/69), done.
remote: Total 782 (delta 32), reused 84 (delta 25), pack-reused 682
Receiving objects: 100% (782/782), 759.52 KiB | 15.50 MiB/s, done.
Resolving deltas: 100% (299/299), done.
Docker params are  --ulimit nofile=4096:4096
Building 'myapp' image using docker build
STEP 1/7: FROM ubi8/nodejs-20:1
STEP 2/7: ADD app-src .
--> 36c4b248c26b
STEP 3/7: USER 0
--> a46d8b1d8f63
STEP 4/7: RUN fix-permissions ./
--> c087f8a238c2
STEP 5/7: USER 1001
--> dbb76744f0be
STEP 6/7: RUN npm install
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated jose@1.28.2: this version is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.

added 785 packages, and audited 786 packages in 28s

107 packages are looking for funding
  run `npm fund` for details

14 vulnerabilities (12 moderate, 1 high, 1 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
npm notice
npm notice New minor version of npm available! 10.2.4 -> 10.5.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.5.0>
npm notice Run `npm install -g npm@10.5.0` to update!
npm notice
--> 5797613b4e3d
STEP 7/7: CMD npm run -d start
phracek commented 6 months ago

[test]

phracek commented 6 months ago

Rebased against master

[test]

phracek commented 6 months ago

Have you tried to use the .build-args files directly from these function? Was that difficult? I did not try that myself, I was just wondering if it possible/sensible way to implement this.

Sure. It does not work from CLI. --ulimit nofile=4096:4096 is called in examples/from-dockerfile which does not support .build-args

zmiklank commented 6 months ago

[test-all]

zmiklank commented 6 months ago

Sure. It does not work from CLI. --ulimit nofile=4096:4096 is called in examples/from-dockerfile which does not support .build-args

I mean adding support for .build-args directly to test scripts. But as I said, I am not sure, if that will make any sense.

phracek commented 6 months ago

Sure. It does not work from CLI. --ulimit nofile=4096:4096 is called in examples/from-dockerfile which does not support .build-args

I mean adding support for .build-args directly to test scripts. But as I said, I am not sure, if that will make any sense.

This would mean, that all containers that are using function ct_test_app_dockerfile will use .build-args as well. I guess, we need to allow this only for relevant containers and to all of them.

zmiklank commented 6 months ago

This would mean, that all containers that are using function ct_test_app_dockerfile will use .build-args as well. I guess, we need to allow this only for relevant containers and to all of them.

OK, let's leave it as it is in this PR. [test]

phracek commented 6 months ago

The failures are expected because the new versions do not have devel repos.

RHEL7 s2i-python-container failure:

[PASSED] for 'mod-wsgi-test-app' test_application_with_user (00:00:05)
 [PASSED] for 'mod-wsgi-test-app' test_application_enable_init_wrapper (00:00:04)
 [FAILED] for 'pin-pipenv-version-test-app' run_s2i_build ()
 [PASSED] for 'micropipenv-requirements-test-app' run_s2i_build ()
 [PASSED] for 'micropipenv-requirements-test-app' test_s2i_usage (00:00:00)

@frenzymadness Do you know where there could be a problem? RHEL7 - Python 3.8

[1m[INFO] Preparing to test pin-pipenv-version-test-app ...

Initialized empty Git repository in /root/sclorg/container-common-scripts/s2i-python-container/3.8/test/pin-pipenv-version-test-app/.git/
[master (root-commit) 52458b7] Sample commit
 4 files changed, 37 insertions(+)
 create mode 100644 .s2i/environment
 create mode 100755 app.sh
 create mode 100644 requirements.txt
 create mode 100644 wsgi.py

[INFO] Building the pin-pipenv-version-test-app application image ...

Sending build context to Docker daemon 56.32 kB

Step 1/10 : FROM rhscl/python-38-rhel7:1
 ---> 5bea623439d7

[ snipped]

Step 9/10 : RUN /usr/libexec/s2i/assemble
 ---> Running in 8d14715764fc

---> Installing application source ...
---> Installing pipenv packaging tool ...
Collecting pipenv==2021.5.29
  Downloading https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl (3.9MB)
Requirement already satisfied, skipping upgrade: pip>=18.0 in ./.local/venvs/pipenv/lib/python3.8/site-packages (from pipenv==2021.5.29) (19.3.1)
Collecting virtualenv-clone>=0.2.5
  Downloading https://files.pythonhosted.org/packages/21/ac/e07058dc5a6c1b97f751d24f20d4b0ec14d735d77f4a1f78c471d6d13a43/virtualenv_clone-0.5.7-py3-none-any.whl
Collecting virtualenv
  Downloading https://files.pythonhosted.org/packages/16/65/0d0bdfdac31e2db8c6d6c18fe1e00236f0dea279f9846f94a9aafa49cfc9/virtualenv-20.25.1-py3-none-any.whl (3.8MB)
Collecting certifi
  Downloading https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl (163kB)
Requirement already satisfied, skipping upgrade: setuptools>=36.2.1 in ./.local/venvs/pipenv/lib/python3.8/site-packages (from pipenv==2021.5.29) (41.6.0)
Collecting platformdirs<5,>=3.9.1
  Downloading https://files.pythonhosted.org/packages/55/72/4898c44ee9ea6f43396fbc23d9bfaf3d06e01b83698bdf2e4c919deceb7c/platformdirs-4.2.0-py3-none-any.whl
Collecting filelock<4,>=3.12.2
  Downloading https://files.pythonhosted.org/packages/81/54/84d42a0bee35edba99dee7b59a8d4970eccdd44b99fe728ed912106fc781/filelock-3.13.1-py3-none-any.whl
Collecting distlib<1,>=0.3.7
  Downloading https://files.pythonhosted.org/packages/8e/41/9307e4f5f9976bc8b7fea0b66367734e8faf3ec84bc0d412d8cfabbb66cd/distlib-0.3.8-py2.py3-none-any.whl (468kB)
Installing collected packages: virtualenv-clone, platformdirs, filelock, distlib, virtualenv, certifi, pipenv
Successfully installed certifi-2024.2.2 distlib-0.3.8 filelock-3.13.1 pipenv-2021.5.29 platformdirs-4.2.0 virtualenv-20.25.1 virtualenv-clone-0.5.7
WARNING: You are using pip version 19.3.1; however, version 24.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
---> Installing dependencies via pipenv ...
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Creating a Pipfile for this project...
Requirements file provided! Importing into Pipfile...
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

⠋ Locking...
Building requirements...

Resolving dependencies...

⠙ Locking...

[snipped]

⠸ Locking...
⠼ Locking...✔ Success! 
Updated Pipfile.lock (9a5f62)!
Installing dependencies from Pipfile.lock (9a5f62)...
An error occurred while installing uwsgi==2.0.24 --hash=sha256:77b6dd5cd633f4ae87ee393f7701f617736815499407376e78f3d16467523afe! Will try again.
Installing initially failed dependencies...
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]:       retcode = do_install(
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]:       do_init(
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]:       do_install_dependencies(
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]:       batch_install(
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/opt/app-root/src/.local/venvs/pipenv/lib64/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Usage:   
[pipenv.exceptions.InstallError]:   pip install [options] <requirement specifier> [package-index-options] ...
[pipenv.exceptions.InstallError]:   pip install [options] -r <requirements file> [package-index-options] ...
[pipenv.exceptions.InstallError]:   pip install [options] [-e] <vcs project url> ...
[pipenv.exceptions.InstallError]:   pip install [options] [-e] <local project path> ...
[pipenv.exceptions.InstallError]:   pip install [options] <archive url/path> ...
[pipenv.exceptions.InstallError]: 
[pipenv.exceptions.InstallError]: --no-use-pep517 error: It is not possible to use --no-use-pep517
[pipenv.exceptions.InstallError]: without setuptools and wheel installed.
ERROR: Couldn't install package: uwsgi
 Package installation failed...
The command '/bin/sh -c /usr/libexec/s2i/assemble' returned a non-zero code: 1
  ERROR: Failed to to build /tmp/tmp.EsyMtJ5mKt/Dockerfile.CeZz

[INFO] TEST FAILED (positive), EXPECTED:0 GOT:1
zmiklank commented 6 months ago

@frenzymadness Do you know where there could be a problem? RHEL7 - Python 3.8

I think this is the same issue as https://github.com/sclorg/s2i-python-container/issues/661.