Remove Legacy Combine LF Workflows - Githubissues

Earth-Information-System / fireatlas

3 stars 1 forks source link

Remove Legacy Combine LF Workflows #28

Closed ranchodeluxe closed 1 month ago

ranchodeluxe commented 1 month ago

Addreses #31

We should be able to use the allfires_gdf from the new Fire_Forward v3 algorithm to write out combined artifacts for all LF layers: perimeter, fireline, newfirepix instead of using these flows.

This PR includes removal of legacy workflows and new workflow to produce outputs

AC:

[x] remove older workflows
[x] figure out why allfires_gdf is missing some key columns we expect to be in the output
[x] write a new workflow for combined that is from allfires_gdf instead of combining across LF outputs

ranchodeluxe commented 1 month ago

Testing this out on JH and works but still needs a couple more things:

the merge process is producing many NaN values that might cause problems during ingestion:

gdf.columns[gdf.isnull().any()].tolist()
['mergeid', 'ftype', 'meanFRP', 't_st', 'fireID', 'geom_counts']

the logic in save_combine_large_fire_layers and save_snapshot_layers should be DRY'd up and reusable