search

NateMietk / modeling-human-ignition

These scripts will take large historical time series stacks of daily climate variables for the US and aggregate to monthly. Steps are unpacking NetCDf, aggregate, mask/clip, and save as GeoTiff Edit Add topics
GNU General Public License v3.0
0 stars 1 forks source link

tile dimension mismatch for batch size > 1 #17

Open joemcglinchy opened 5 years ago

joemcglinchy commented 5 years ago

It looks like the tiles are not all the same size which causes issues for batch sizes > 1 during training. We should:

1) write out the tile rasters (e.g., t1.tif) with some value to then load into a GIS to see where they line up spatially. Initial thought is they should be on the CONUS border 2) do a data re-write such that all tiles are the same size, via zero-padding. The land mask will be zero-padded the same way so if all 'bad' tiles are on the border it shouldn't matter for masked loss calculations.

joemcglinchy commented 5 years ago

I think we're good. Looks like the tiles with those dimensions comprise the border tiles. See image below (yellow tiles have dimensions != 448x448). Next step is to do the re-write. I'm happy to tackle that unless someone else wants to do that! I am also planning to run through a model iteration test of doing it 'on the fly'.

image