nismod / open-gira

Open-data Global Infrastructure Risk/Resilience Analysis
https://nismod.github.io/open-gira/
MIT License
12 stars 3 forks source link

Intermittent failure of test_estimate_wind_fields.py #136

Open thomas-fred opened 11 months ago

thomas-fred commented 11 months ago

results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc does not match reference some of the time.

=================================== FAILURES ===================================
__________________________ test_estimate_wind_fields ___________________________
[gw1] linux -- Python 3.10.8 /home/runner/micromamba-root/envs/open-gira/bin/python

    def test_estimate_wind_fields():
>       runner.run_snakemake_test(
            "estimate_wind_fields",
            (
                "results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc",
            )
        )

...

<xarray.Dataset>
Dimensions:         (event_id: 29, latitude: 36, longitude: 49)
Coordinates:
  * event_id        (event_id) object '2001235N18296' ... '2022257N16312'
  * latitude        (latitude) float64 16.47 16.57 16.67 ... 19.77 19.87 19.97
  * longitude       (longitude) float64 -68.74 -68.64 -68.54 ... -64.04 -63.94
Data variables:
    spatial_ref     int64 0
    max_wind_speed  (event_id, latitude, longitude) float64 5.642 ... 9.801

<xarray.Dataset>
Dimensions:         (event_id: 29, latitude: 36, longitude: 49)
Coordinates:
  * event_id        (event_id) object '2001235N18296' ... '2022257N16312'
  * latitude        (latitude) float64 16.47 16.57 16.67 ... 19.77 19.87 19.97
  * longitude       (longitude) float64 -68.74 -68.64 -68.54 ... -64.04 -63.94
Data variables:
    spatial_ref     int64 0
    max_wind_speed  (event_id, latitude, longitude) float64 5.642 ... 9.801

----------------------------- Captured stderr call -----------------------------
Config file config/config.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Conda environments: ignored
Job stats:
job                     count    min threads    max threads
--------------------  -------  -------------  -------------
estimate_wind_fields        1              1              1
total                       1              1              1

Select jobs to execute...

[Mon Jul 24 13:29:43 2023]
rule estimate_wind_fields:
    input: results/power/by_country/PRI/storms/IBTrACS/tracks.geoparquet, results/power/by_country/PRI/storms/wind_grid.tiff, results/power/by_country/PRI/storms/surface_roughness.tiff
    output: results/power/by_country/PRI/storms/IBTrACS/plots, results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc, results/power/by_country/PRI/storms/IBTrACS/downscale_factors.png
    jobid: 0
    reason: Missing output files: results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc
    wildcards: OUTPUT_DIR=results, COUNTRY_ISO_A3=PRI, STORM_SET=IBTrACS
    resources: tmpdir=/tmp

2023-07-24 13:29:46,497 5130 tmp1sgbgg3b.estimate_wind_fields.py Reading tracks
2023-07-24 13:29:46,579 5130 tmp1sgbgg3b.estimate_wind_fields.py 
                     year  month  ...   name                    geometry
time_utc                          ...                                   
2001-08-22 12:00:00  2001      8  ...   DEAN  POINT (-64.30000 17.90000)
...
2022-09-19 12:00:00  2022      9  ...  FIONA  POINT (-68.90000 18.80000)

[201 rows x 14 columns]
2023-07-24 13:29:46,579 5130 tmp1sgbgg3b.estimate_wind_fields.py Reading wind evaluation grid
2023-07-24 13:29:46,615 5130 tmp1sgbgg3b.estimate_wind_fields.py 
<xarray.DataArray (band: 1, y: 36, x: 49)>
[1764 values with dtype=uint8]
Coordinates:
  * band         (band) int64 1
  * x            (x) float64 -68.74 -68.64 -68.54 ... -64.14 -64.04 -63.94
  * y            (y) float64 16.47 16.57 16.67 16.77 ... 19.67 19.77 19.87 19.97
    spatial_ref  int64 0
Attributes:
    AREA_OR_POINT:  Area
    scale_factor:   1.0
    add_offset:     0.0
2023-07-24 13:29:46,615 5130 tmp1sgbgg3b.estimate_wind_fields.py Calculating downscaling factors from surface roughness raster
2023-07-24 13:29:46,628 5130 tmp1sgbgg3b.estimate_wind_fields.py Mapping downscaling factors and saving to disk
2023-07-24 13:29:46,862 5130 tmp1sgbgg3b.estimate_wind_fields.py Estimating wind fields for 29 storm tracks
2023-07-24 13:29:46,868 5130 tmp1sgbgg3b.estimate_wind_fields.py 2001235N18296
...
2023-07-24 13:29:55,939 5130 tmp1sgbgg3b.estimate_wind_fields.py 2022257N16312
2023-07-24 13:29:56,376 5130 tmp1sgbgg3b.estimate_wind_fields.py Saving maximum wind speeds to disk
2023-07-24 13:29:56,416 5130 tmp1sgbgg3b.estimate_wind_fields.py Done estimating wind fields
[Mon Jul 24 13:29:56 2023]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2023-07-24T132941.285084.snakemake.log

>>> Compare files:
/tmp/tmpudvro210/workdir/results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc
tests/integration/estimate_wind_fields/expected/results/power/by_country/PRI/storms/IBTrACS/max_wind_field.nc
=============================== warnings summary ===============================
tests/integration/runner.py:17
tests/integration/runner.py:17
tests/integration/test_electricity_grid_damages.py::test_electricity_grid_damages

=========================== short test summary info ============================
FAILED tests/integration/test_estimate_wind_fields.py::test_estimate_wind_fields - AssertionError
============= 1 failed, 15 passed, 2 skipped, 3 warnings in 48.63s =============