Distgen errors

[zmiklank]

Distgen is throwing errors while generating content for s2i-python container.

Logs can be seen when running localy (make generate-all) or in CI[1].

[1] https://github.com/sclorg/s2i-python-container/actions/runs/4383184504/jobs/7673113942

[frenzymadness]

make generate-all tries to generate all combinations but only a subset of them is defined in multispec file. The errors are just complains that a specific combination is not defined in the multispec. Does that cause any troubles?

[frenzymadness]

For example, manifest.yml says that we should generate Dockerfile.centos7 for all Pythons but in multispec, there is no combination of Python 3.11 on Centos 7.

[zmiklank]

Thanks for confirmation, I also thought about something similar. However the log looks like something really bad had happened.

dg: CRITICAL: This combination is not included in matrix section
[ERROR] distgen failed: Command '['dg', '--multispec', 'specs/multispec.yml', '--template', 'src/Dockerfile.template', '--distro', 'rhel-9-x86_64.yaml', '--multispec-selector', 'version=3.11', '--output', '3.11/Dockerfile.rhel9']'

I just think that this issue could confuse somebody else like it confused me this time. I do not see as much to the distgen generation, so if you could see some easy fix that would be great. If not, we can either consider this as correct behavior, or leave this issue here and I will get to it eventually.

What do you think?

[frenzymadness]

dg: CRITICAL: This combination is not included in matrix section - this part is produced by the distgen itself, we can silence it but it would mean that we won't see any errors in the future when they might be useful for debugging. [ERROR] distgen failed: Command '['dg', '--multispec', 'specs/multispec.yml', '--template', 'src/Dockerfile.template', '--distro', 'rhel-9-x86_64.yaml', '--multispec-selector', 'version=3.11', '--output', '3.11/Dockerfile.rhel9']' - this comes from the generator itself but only the [ERROR] distgen failed: is adjustable because the rest is in the exception raised by the failed call of check_output.

If distgen has a special error code for the cases of missing combinations in the matrix, we can update generator to ignore that specific error. Or we can try to workaround it by matching the error message dg produces but that might be flaky.

[zmiklank]

If distgen has a special error code for the cases of missing combinations in the matrix, we can update generator to ignore that specific error.

I like this. But I would say this is of not very high priority. Thanks for the analysis.

[frenzymadness]

I've prepared an implementation for distgen which should allow us to safely ignore some types of errors from distgen.

[frenzymadness]

So the distgen part is done and released: https://github.com/devexp-db/distgen/pull/130 . @befeleme is this something you'd be interested in? This has to be implemented in the common scripts so you might gain some hands-on experience with submodule synchronization and testing of changes there. The change itself should not be complex.

[befeleme]

I'll look at this, thank you!

[befeleme]

What's the desired action generator should do when encountering exit code 2? I'm thinking of suppressing the error message, so this would not be printed at all: [ERROR] distgen failed: Command '['dg', '--multispec', 'specs/multispec.yml', '--template', 'src/Dockerfile.template', '--distro', 'rhel-9-x86_64.yaml', '--multispec-selector', 'version=3.11', '--output', '3.11/Dockerfile.rhel9']'. Would you like to have some other message printed instead?

[frenzymadness]

What's the desired action generator should do when encountering exit code 2? I'm thinking of suppressing the error message, so this would not be printed at all: [ERROR] distgen failed: Command '['dg', '--multispec', 'specs/multispec.yml', '--template', 'src/Dockerfile.template', '--distro', 'rhel-9-x86_64.yaml', '--multispec-selector', 'version=3.11', '--output', '3.11/Dockerfile.rhel9']'. Would you like to have some other message printed instead?

I think we discussed it yesterday so just FTR I believe that suppressing the error is all we need.

[zmiklank]

This feature is AFAIK already implemented and works well. Thanks all for the work on this. Closing.