ValueError for positive and negative calibration cycles

[ulysses78]

When working with the new pollyxt_pipeline version 1.12.0 it sometimes gives me the following error:

ValueError

  Positive and negative calibration cycles have different lengths: 7 vs 6

  at scc_env/lib64/python3.8/site-packages/pollyxt_pipelines/polly_to_scc/scc_netcdf.py:240 in create_scc_calibration_netcdf
      236│     end_negative = pf.depol_cal_angle.shape[0] - 3
      237│     negative_length = end_negative - start_negative
      238│
      239│     if positive_length != negative_length:
    → 240│         raise ValueError(
      241│             f"Positive and negative calibration cycles have different lengths: {positive_length} vs {negative_length}"
      242│         )
      243│
      244│     # Create Dimensions. (mandatory)

How can we prevent this error?

[ulysses78]

@thgeorgiou

[ulysses78]

It is not possible for me to assign someone.

[ulysses78]

or add a bug label

[thgeorgiou]

Hello! Could you send me a sample file to check?

[ulysses78]

Here: https://owncloud.gwdg.de/index.php/f/2209941685 you can find two nc-files. Here is what I get when trying to convert from those two nc files into the scc-fileformat. After the mentioned error message, the next files will be created but not uploaded and another error message occurs!

Building repository... 
Created file with measurement ID 20221202lei0000 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0000.nc (2022-12-02 00:00 - 2022-12-02 01:00)
Created file with measurement ID 20221202lei0100 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0100.nc (2022-12-02 01:00 - 2022-12-02 02:00)
Created file with measurement ID 20221202lei0200 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0200.nc (2022-12-02 02:00 - 2022-12-02 03:00)
Created file with measurement ID 20221202lei0300 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0300.nc (2022-12-02 03:00 - 2022-12-02 04:00)
Created file with measurement ID 20221202lei0400 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0400.nc (2022-12-02 04:00 - 2022-12-02 05:00)
Created file with measurement ID 20221202lei0500 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0500.nc (2022-12-02 05:00 - 2022-12-02 05:59)

ValueError

  Positive and negative calibration cycles have different lengths: 7 vs 6

  at scc_env/lib64/python3.8/site-packages/pollyxt_pipelines/polly_to_scc/scc_netcdf.py:240 in create_scc_calibration_netcdf
      236│     end_negative = pf.depol_cal_angle.shape[0] - 3
      237│     negative_length = end_negative - start_negative
      238│
      239│     if positive_length != negative_length:
    → 240│         raise ValueError(
      241│             f"Positive and negative calibration cycles have different lengths: {positive_length} vs {negative_length}"
      242│         )
      243│
      244│     # Create Dimensions. (mandatory)
Building repository...
Created file with measurement ID 20221202lei0600 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0600.nc (2022-12-02 06:00 - 2022-12-02 07:00)
Created file with measurement ID 20221202lei0700 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0700.nc (2022-12-02 07:00 - 2022-12-02 08:00)
Created file with measurement ID 20221202lei0800 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0800.nc (2022-12-02 08:00 - 2022-12-02 09:00)
Created file with measurement ID 20221202lei0900 at /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0900.nc (2022-12-02 09:00 - 2022-12-02 09:17)

Done!
Uploading...
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0000.nc: File already exists
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0100.nc: File already exists
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0200.nc: File already exists
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0300.nc: File already exists
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0400.nc: File already exists
Error while uploading /pollyhome/polly2scc/thanasis_scc/files_nc/scc_data/lacros/20221202lei0500.nc: File already exists
Uploading files... ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━  60% 0:00:02

  AttributeError

  NetCDF: Attribute not found

  at src/netCDF4/_netCDF4.pyx:1965 in netCDF4._netCDF4._ensure_nc_success

[ulysses78]

sorry, wrong link. it has to be: https://owncloud.gwdg.de/index.php/s/Sd9OHDbxLoDzjrj

[ulysses78]

I just tested the uploading procedure... it seems, that every single converted file daytime and nighttime file can be uploaded, but the depol_calib files can not! This is where the error message seems to pop up. So only depol-calib files can not be uploaded at the moment. We need your assistance here ;-)

[thgeorgiou]

Hello! I made some changes in 79e2cb4 and pushed a new version (1.12.1). Could you check if it works out in your case?

PS: If you want, send me your locations.ini so I can add your station to the project so that it works out of the box!

[ulysses78]

Yes. Thanks! It is working again! No error messages and the upload of the depol-calib files is working as well. Our locations.ini file I have already updated manually - otherwise the conversion script wouldn't work/accept the locations.ini.

[ulysses78]

So I guess, this issue can be closed.

[thgeorgiou]

Yes. Thanks! It is working again! No error messages and the upload of the depol-calib files is working as well.

Excellent!

Our locations.ini file I have already updated manually - otherwise the conversion script wouldn't work/accept the locations.ini.

If you parameters are somewhat standard, I can add your information to the default locations.ini so that it works without manual intervention.