tobac-project / tobac

Tracking and object-based analysis of clouds
BSD 3-Clause "New" or "Revised" License
99 stars 53 forks source link

Add matrix CI Tests on multiple Python versions and OS versions #353

Closed freemansw1 closed 10 months ago

freemansw1 commented 11 months ago

This PR adds matrix testing (a matrix of Linux, MacOS, Windows for Pythons 3.8, 3.9, 3.10, 3.11). Right now, this is configured to run on the primary tobac repo (i.e., this one) only, and not on fork repos to save computation time.

Note that this has indicated a bug/issue in testing with int32 vs int64 datatypes on Windows. I have a windows computer at home; I will have to debug there.

codecov[bot] commented 11 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (5f096e3) 56.68% compared to head (b7739d5) 56.70%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## RC_v1.5.x #353 +/- ## ============================================= + Coverage 56.68% 56.70% +0.02% ============================================= Files 19 19 Lines 3426 3428 +2 ============================================= + Hits 1942 1944 +2 Misses 1484 1484 ``` | [Flag](https://app.codecov.io/gh/tobac-project/tobac/pull/353/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/tobac-project/tobac/pull/353/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project) | `56.70% <100.00%> (+0.02%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/tobac-project/tobac/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project) | Coverage Δ | | |---|---|---| | [tobac/testing.py](https://app.codecov.io/gh/tobac-project/tobac/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project#diff-dG9iYWMvdGVzdGluZy5weQ==) | `95.55% <100.00%> (ø)` | | | [tobac/utils/general.py](https://app.codecov.io/gh/tobac-project/tobac/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tobac-project#diff-dG9iYWMvdXRpbHMvZ2VuZXJhbC5weQ==) | `71.14% <100.00%> (+0.19%)` | :arrow_up: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

freemansw1 commented 11 months ago

I fixed the Windows bugs through keeping data types constant. Happy to wait for a re-review to merge if you want another look at it @JuliaKukulies @w-k-jones

w-k-jones commented 11 months ago

I'm happy to merge straight away, I don't think the fix requires a re-review as it looks like a pretty straightforward (and sensible) change

w-k-jones commented 11 months ago

Actually, as a workflow change, should this be merged into RC_v1.5.x or main?

Edit: it has some changes from RC_v1.5.x included, so I think it's best to keep the merge target as is

freemansw1 commented 10 months ago

Looks like #293 broke windows compatibility... will need to look into this before merging.

JuliaKukulies commented 10 months ago

Looks like #293 broke windows compatibility... will need to look into this before merging.

Could that also be related to the change of dtypes in the feature detection/segmentation dataframes? What was the actual problem before you set these to constant in utils.general.combine_feature_dataframes ? Is that a thing we should consider doing in all functions that somehow modify the pandas dataframes?

freemansw1 commented 10 months ago

Looks like #293 broke windows compatibility... will need to look into this before merging.

Could that also be related to the change of dtypes in the feature detection/segmentation dataframes? What was the actual problem before you set these to constant in utils.general.combine_feature_dataframes ? Is that a thing we should consider doing in all functions that somehow modify the pandas dataframes?

At first glance on my windows machine, it's not a datatype issue... rather the dataframe comes up empty (now, is this itself a result of a data type issue? Perhaps). I will work to debug here.

freemansw1 commented 10 months ago

Looks like #293 broke windows compatibility... will need to look into this before merging.

Could that also be related to the change of dtypes in the feature detection/segmentation dataframes? What was the actual problem before you set these to constant in utils.general.combine_feature_dataframes ? Is that a thing we should consider doing in all functions that somehow modify the pandas dataframes?

At first glance on my windows machine, it's not a datatype issue... rather the dataframe comes up empty (now, is this itself a result of a data type issue? Perhaps). I will work to debug here.

On second thought, I'm going to strongly bet that this is a time issue. Ugh. Times are going to be the death of me.

freemansw1 commented 10 months ago

Okay, this was actually an amazing issue to fix. On Windows, the lines in the test dataset making that are: https://github.com/tobac-project/tobac/blob/5f096e3e66da07cfd48a6a4588b993fc1263a0c2/tobac/testing.py#L435

result in a float output (with a real <1 value even!). When they are converted to xarray, xarray preserves the milliseconds contained in these values, but our Iris feature detection path does not use them, resulting in incompatible times. This was a neat issue to find actually! Anyway it's resolved now!

JuliaKukulies commented 10 months ago

Great job @freemansw1! Do you want to go ahead and merge this?