containers / toolbox

Tool for interactive command line environments on Linux
https://containertoolbx.org/
Apache License 2.0
2.38k stars 208 forks source link

Optimize 'enter' and 'run' for a container getting initialized #1495

Closed debarshiray closed 1 month ago

debarshiray commented 1 month ago

Currently, the enter and run commands poll at one second intervals to check if the Toolbx container's entry point has created the initialization stamp file to indicate that the container has been initialized. This came from the POSIX shell implementation [1], where it was relatively easier to poll than to use inotify(7) to monitor the file system.

The problem with polling is that the interval is always going to be either too short and waste resources or too long and cause delays. The current one second interval is sufficiently long to add a noticeable delay to the enter and run commands.

It will be better to use inotify(7) to monitor the file system, which is quite easy to do with the Go implementation, so that the commands can proceed as soon as the initialization stamp file is available, instead of waiting for the polling interval to pass.

There's a fallback to polling, as before, when the operating system is suffering from a shortage of resources needed for inotify(7). This code path can be forced through the TOOLBX_RUN_USE_POLLING environment variable for testing. Setting this environment variable disables some code to ensure that the polling ticker is actually used, because, otherwise, the race between the creation and detection of the initialization stamp file makes it difficult to test the fallback.

[1] Commit d3e0f3df06d3f5ac https://github.com/containers/toolbox/commit/d3e0f3df06d3f5ac https://github.com/containers/toolbox/pull/305

https://github.com/containers/toolbox/issues/1070

softwarefactory-project-zuul[bot] commented 1 month ago

Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/e071ec50213946ed8fa6e0099e6c5eed

:heavy_check_mark: unit-test SUCCESS in 6m 42s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 35s :heavy_check_mark: unit-test-restricted SUCCESS in 5m 53s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 35m 29s :heavy_check_mark: system-test-fedora-40 SUCCESS in 32m 41s :heavy_check_mark: system-test-fedora-39 SUCCESS in 33m 17s :heavy_check_mark: system-test-fedora-38 SUCCESS in 32m 10s

softwarefactory-project-zuul[bot] commented 1 month ago

Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/7e3d069613934350a8dc9a400e320f4c

:heavy_check_mark: unit-test SUCCESS in 6m 31s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 04s :heavy_check_mark: unit-test-restricted SUCCESS in 5m 37s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 35m 42s :heavy_check_mark: system-test-fedora-40 SUCCESS in 34m 15s :heavy_check_mark: system-test-fedora-39 SUCCESS in 33m 58s :heavy_check_mark: system-test-fedora-38 SUCCESS in 33m 48s

softwarefactory-project-zuul[bot] commented 1 month ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/cd7d1c901d924802b7b19cb02ef9fdf3

:heavy_check_mark: unit-test SUCCESS in 6m 48s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 51s :heavy_check_mark: unit-test-restricted SUCCESS in 6m 00s :x: system-test-fedora-rawhide FAILURE in 35m 58s :x: system-test-fedora-40 FAILURE in 34m 15s :x: system-test-fedora-39 FAILURE in 32m 39s :heavy_check_mark: system-test-fedora-38 SUCCESS in 33m 18s

softwarefactory-project-zuul[bot] commented 1 month ago

Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/4e820e8872914a35be2e8e89384bb5f2

:heavy_check_mark: unit-test SUCCESS in 7m 19s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 57s :heavy_check_mark: unit-test-restricted SUCCESS in 6m 11s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 36m 29s :heavy_check_mark: system-test-fedora-40 SUCCESS in 33m 46s :heavy_check_mark: system-test-fedora-39 SUCCESS in 34m 10s :heavy_check_mark: system-test-fedora-38 SUCCESS in 33m 25s

softwarefactory-project-zuul[bot] commented 1 month ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/31353efeaf6a40719897ddb35175c13e

:heavy_check_mark: unit-test SUCCESS in 7m 14s :x: unit-test-migration-path-for-coreos-toolbox POST_FAILURE in 3m 47s :heavy_check_mark: unit-test-restricted SUCCESS in 6m 05s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 38m 39s :heavy_check_mark: system-test-fedora-40 SUCCESS in 36m 53s :heavy_check_mark: system-test-fedora-39 SUCCESS in 35m 59s :heavy_check_mark: system-test-fedora-38 SUCCESS in 36m 16s

debarshiray commented 1 month ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/31353efeaf6a40719897ddb35175c13e

unit-test-migration-path-for-coreos-toolbox POST_FAILURE in 3m 47s

I will waive that POST_FAILURE because the only change between the last two runs was this addition to test/system/104-run/bats:

+  assert_output --partial "Handling polling tick"
softwarefactory-project-zuul[bot] commented 1 month ago

Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/dce365c2b2d04ce691e49f973c0925bb

:heavy_check_mark: unit-test SUCCESS in 6m 16s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 5m 35s :heavy_check_mark: unit-test-restricted SUCCESS in 6m 13s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 35m 51s :heavy_check_mark: system-test-fedora-40 SUCCESS in 33m 25s :heavy_check_mark: system-test-fedora-39 SUCCESS in 33m 55s :heavy_check_mark: system-test-fedora-38 SUCCESS in 33m 42s