ARPA-SIMC / arkimet

A set of tools to organize, archive and distribute data files.
Other
15 stars 5 forks source link

build errors on fedora 40 #337

Closed brancomat closed 1 month ago

brancomat commented 1 month ago

I'm a bit puzzled because this is basically issue #331 that was fixed in branch cpp17 that was merged in master that at the market my father bought

Full log: https://simc.arpae.it/moncic-ci/arkimet/202410080827/master/fedora40/build.log

Relevant bits:

2024-10-08 11:23:28 INFO system.fedora40 stdout: ======================================================================
2024-10-08 11:23:28 INFO system.fedora40 stdout: FAIL: test_extra (tests.test_arki_mergeconf.TestArkiMergeconf.test_extra)
2024-10-08 11:23:28 INFO system.fedora40 stdout: ----------------------------------------------------------------------
2024-10-08 11:23:28 INFO system.fedora40 stdout: Traceback (most recent call last):
2024-10-08 11:23:28 INFO system.fedora40 stdout:   File "/root/rpmbuild/BUILD/arkimet-1.51-1/python/tests/test_arki_mergeconf.py", line 121, in test_extra
2024-10-08 11:23:28 INFO system.fedora40 stdout:     self.assertEqual(out.splitlines(), [
2024-10-08 11:23:28 INFO system.fedora40 stdout: AssertionError: Lists differ: ['[/t[60 chars]((-60 30, -60 72, 42 72, 42 30, -60 30))', 'fo[121 chars]ile'] != ['[/t[60 chars]((-60.0000000000000000 30.0000000000000000, -6[291 chars]ile']
2024-10-08 11:23:28 INFO system.fedora40 stdout: 
2024-10-08 11:23:28 INFO system.fedora40 stdout: First differing element 1:
2024-10-08 11:23:28 INFO system.fedora40 stdout: 'boun[15 chars]((-60 30, -60 72, 42 72, 42 30, -60 30))'
2024-10-08 11:23:28 INFO system.fedora40 stdout: 'boun[15 chars]((-60.0000000000000000 30.0000000000000000, -6[160 chars]00))'
2024-10-08 11:23:28 INFO system.fedora40 stdout: 
2024-10-08 11:23:28 INFO system.fedora40 stdout:   ['[/tmp/tmp.Is00x5SuFJ/inbound/test.grib1]',
2024-10-08 11:23:28 INFO system.fedora40 stdout: -  'bounding = POLYGON ((-60 30, -60 72, 42 72, 42 30, -60 30))',
2024-10-08 11:23:28 INFO system.fedora40 stdout: +  'bounding = POLYGON ((-60.0000000000000000 30.0000000000000000, '
2024-10-08 11:23:28 INFO system.fedora40 stdout: +  '-60.0000000000000000 72.0000000000000000, 42.0000000000000000 '
2024-10-08 11:23:28 INFO system.fedora40 stdout: +  '72.0000000000000000, 42.0000000000000000 30.0000000000000000, '
2024-10-08 11:23:28 INFO system.fedora40 stdout: +  '-60.0000000000000000 30.0000000000000000))',
2024-10-08 11:23:28 INFO system.fedora40 stdout:    'format = grib',
2024-10-08 11:23:28 INFO system.fedora40 stdout:    'name = /tmp/tmp.Is00x5SuFJ/inbound/test.grib1',
2024-10-08 11:23:28 INFO system.fedora40 stdout:    'path = /tmp/tmp.Is00x5SuFJ/inbound/test.grib1',
2024-10-08 11:23:28 INFO system.fedora40 stdout:    'type = file']
dcesari commented 1 month ago

It looks like there are some syntax errors in the regular expression here:

https://github.com/ARPA-SIMC/arkimet/blob/57d42abb9bf109395d798ce39f0b593c91aeaafa/python/tests/test_arki_mergeconf.py#L147

a missing ) and missing ?'s after parentheses!?

spanezz commented 1 month ago

It looks like there are some syntax errors in the regular expression here: Indeed, you're right (I couldn't reproduce this issue so it was hard to test the fix).

I've abstracted the bit of regexp in question and made it reusable. I hope that this time, if I didn't spell it right, at least it can be fixed in a single place

brancomat commented 1 month ago

now, different errors: https://simc.arpae.it/moncic-ci/arkimet/last/master/fedora40/build.log

apparently there's also an issue with moncic-ci not downloading the latest master, tracked by @spanezz somewhere around https://github.com/ARPA-SIMC/moncic-ci/blob/main/moncic/build/arpa.py#L88

edigiacomo commented 1 month ago

Now a different error (again) on Fedora 40 only:

2024-10-18 15:43:56 INFO system.fedora40 stdout: Scan a well-known grib file, with no padding between messages ... ok
2024-10-18 15:43:56 INFO system.fedora40 stdout: test_cosmo_nudging (tests.test_scan_grib.TestScanGrib.test_cosmo_nudging)
2024-10-18 15:43:56 INFO system.fedora40 stdout: Check scanning COSMO nudging timeranges ... ecCodes Version:       2.38.0
2024-10-18 15:43:56 INFO system.fedora40 stdout: Definition files path: /usr/share/eccodes-simc/definitions:/usr/share/eccodes/definitions
2024-10-18 15:43:56 INFO system.fedora40 stdout: ECCODES ERROR   :  Parser include: Could not resolve 'grib2/template.3.shape_of_the_earth.def' (included in /usr/share/eccodes-simc/definitions/grib2/local/200/template.3.32768.def)
2024-10-18 15:43:56 INFO system.fedora40 stdout: ecCodes assertion failed: `0' in /builddir/build/BUILD/eccodes-2.38.0-Source/src/grib_context.cc:220
2024-10-18 15:43:56 INFO system.fedora40 stdout: Fatal Python error: Aborted
edigiacomo commented 1 month ago

The tests on Fedora 38 (eccodes v2.33.0) ran successfully. I suspect that eccodes v2.38.0 searches for grib2/template.3.shape_of_the_earth.def in eccodes-simc definitions only:

# Fedora 38
$ rpm -qi eccodes | grep -i version
Version     : 2.33.0
$ grep -R template.3.shape_of_the_earth.def /usr/share/eccodes-simc/definitions/grib2/local/200/template.3.32768.def
    include "grib2/template.3.shape_of_the_earth.def";
$ find /usr/share/eccodes* -name template.3.shape_of_the_earth.def
/usr/share/eccodes/definitions/grib2/template.3.shape_of_the_earth.def
/usr/share/eccodes/definitions/grib3/template.3.shape_of_the_earth.def

# Fedora 40
$ rpm -qi eccodes | grep -i version
Version     : 2.38.0
$ grep -R template.3.shape_of_the_earth.def /usr/share/eccodes-simc/definitions/grib2/local/200/template.3.32768.def
    include "grib2/template.3.shape_of_the_earth.def";
$ find /usr/share/eccodes* -name template.3.shape_of_the_earth.def
/usr/share/eccodes/definitions/grib2/templates/template.3.shape_of_the_earth.def
/usr/share/eccodes/definitions/grib3/template.3.shape_of_the_earth.def
edigiacomo commented 1 month ago

After reading the formatted text, I realized that there is an extra directory (templates): /usr/share/eccodes/definitions/grib2/templates/template.3.shape_of_the_earth.def

edigiacomo commented 1 month ago

The error is now different, but the build is successful on Rocky 8 only, hence I close this issue and open a new one.