search

JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA
8 stars 2 forks source link

adpupa prepbufr yaml dimension fix #1438

Closed PraveenKumar-NOAA closed 5 months ago

PraveenKumar-NOAA commented 6 months ago

Description

This PR updates dimension of variables into the adpupa prepbufr yaml and removes extra copies of the yaml/nc files for this data type.

The following two files were modified for the dimension updates:

The following two files were removed:

The following file is also modified to remove the unit test for the adpupa_prepbufr.yaml:

Dependencies

None

Impact

None

PatNichols commented 6 months ago

@PraveenKumar-NOAA I am not sure you can see the output from your ctests since cdash is failing at the moment: The following tests FAILED:

1215 | 115 - test_iodaconv_bufr_ncep_prepbufr_adpupa (Failed) 1216 | 116 - test_iodaconv_bufr_ncep_prepbufr_adpupa_api (Failed)

PraveenKumar-NOAA commented 6 months ago

@PraveenKumar-NOAA I am not sure you can see the output from your ctests since cdash is failing at the moment:

The following tests FAILED: 1215 | 115 - test_iodaconv_bufr_ncep_prepbufr_adpupa (Failed) 1216 | 116 - test_iodaconv_bufr_ncep_prepbufr_adpupa_api (Failed)

@PatNichols thanks, I will check once HERA is up for work.

PraveenKumar-NOAA commented 6 months ago

@PraveenKumar-NOAA I am not sure you can see the output from your ctests since cdash is failing at the moment:

The following tests FAILED: 1215 | 115 - test_iodaconv_bufr_ncep_prepbufr_adpupa (Failed) 1216 | 116 - test_iodaconv_bufr_ncep_prepbufr_adpupa_api (Failed)

@PatNichols thanks, I will check once HERA is up for work.

@PatNichols this problem is arising after I trimmed the input bufr file. Is it OK to proceed with the original ADPUPA.prepbufr file, which leads to the output NC file > 1 Mb?

PraveenKumar-NOAA commented 6 months ago

@emilyhcliu @rmclaren please review this PR.

PatNichols commented 6 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

PraveenKumar-NOAA commented 6 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

@PatNichols No, I tried to reduce the size for different capacity, but it didn't work. We have to keep the same size.

PatNichols commented 6 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

@PatNichols No, I tried to reduce the size for different capacity, but it didn't work. We have to keep the same size.

@PraveenKumar-NOAA Does it fail the test with a smaller file or does the reduction fail to produce a usable file?

PraveenKumar-NOAA commented 6 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

@PatNichols No, I tried to reduce the size for different capacity, but it didn't work. We have to keep the same size.

@PraveenKumar-NOAA Does it fail the test with a smaller file or does the reduction fail to produce a usable file?

@PatNichols test is failed with a smaller file.

PatNichols commented 5 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

@PatNichols No, I tried to reduce the size for different capacity, but it didn't work. We have to keep the same size.

@PraveenKumar-NOAA Does it fail the test with a smaller file or does the reduction fail to produce a usable file?

@PatNichols test is failed with a smaller file.

@PraveenKumar-NOAA Is it the case that reducing the netcdf file require the bufr file to be reduced and that is not easy?

PraveenKumar-NOAA commented 5 months ago

@PraveenKumar-NOAA is it possible to reduce the test file size?

@PatNichols No, I tried to reduce the size for different capacity, but it didn't work. We have to keep the same size.

@PraveenKumar-NOAA Does it fail the test with a smaller file or does the reduction fail to produce a usable file?

@PatNichols test is failed with a smaller file.

@PraveenKumar-NOAA Is it the case that reducing the netcdf file require the bufr file to be reduced and that is not easy?

@PatNichols yes, that is true.

PraveenKumar-NOAA commented 5 months ago

@PatNichols do you have any other question/ issue?

PatNichols commented 5 months ago

@PraveenKumar-NOAA We are just looking at the file sizes. I suppose it's not as big as an issue since you are removing on ~1.5 Mb and adding another. @fcvdb @BenjaminRuston What do you guys think?