tldr; this is just creating extra files and slowing down our s3 puts/gets with no real benefit.
Currently when you run the grid-and-attrs job on a dataset it produces the output and splits the data by 1km MGRS grids into individual files per grid. It's often quite nice to be able to select only the buildings within an MGRS grid, but thus far having the buildings partitioned by grid isn't helping anything in terms of the physical files. We probably don't want a single large file per state, so we'll still want some way to partition data, but 1km grids are likely too small and it's making things harder rather than easier.
tldr; this is just creating extra files and slowing down our s3 puts/gets with no real benefit.
Currently when you run the
grid-and-attrsjob on a dataset it produces the output and splits the data by 1km MGRS grids into individual files per grid. It's often quite nice to be able to select only the buildings within an MGRS grid, but thus far having the buildings partitioned by grid isn't helping anything in terms of the physical files. We probably don't want a single large file per state, so we'll still want some way to partition data, but 1km grids are likely too small and it's making things harder rather than easier.