search

RADAR-base / radar-output-restructure

Reads avro files in HDFS and outputs json or csv per topic per user in local file system
Apache License 2.0
1 stars 0 forks source link

Configuration bug #552

Closed Bdegraaf1234 closed 3 months ago

Bdegraaf1234 commented 3 months ago

This pull request is to fix an apparent bug (#551) in the output restructure configuration process.

I added a test but cannot capture the problem in a proper unit test so added an integration test.

Bdegraaf1234 commented 3 months ago

in 6814c16f21a45032344e23ec0bfdbff93ecb86e3 You can see a log message which indicates that the two levels of configuration are the same for the unittest added in this pull request (recordPathFactoryTest.kt, as well as the other unit tests that I checked) but NOT for the integration test.

image

https://github.com/RADAR-base/radar-output-restructure/actions/runs/8482449189/job/23241706408?pr=552#step:6:420

https://github.com/RADAR-base/radar-output-restructure/actions/runs/8482449189/job/23241706408?pr=552#step:6:589

Bdegraaf1234 commented 3 months ago

@pvannierop and I looked more closely at this and found the issue was located in the PathConfig.createFactory() method, which in its current implementation could not pass any configuration to the created factory.path property.