Closed mileswwatkins closed 5 years ago
I've Dockerized, but running within Docker Compose results in a crash within the ogr2ogr
line. This appears to be related to hardware resources, and happens even when I've allocated 12 GB RAM and 2 CPU cores to Docker Machine. It's likely that the 4 GB RAM limit within CircleCI won't pass this threshold, but I'm going to check this.
make-tiles_1 | ./make-tiles.sh: line 65: 8 Killed ogr2ogr -clipsrc ./data/cb_2017_us_nation_5m.shp -where "GEOID NOT LIKE '%ZZZ'" -f GeoJSON ./data/sld-clipped.geojson ./data/sld.geojson
Bleh, yeah, I get this in the CircleCI output:
./make-tiles.sh: line 65: 370 Killed ogr2ogr -clipsrc ./data/cb_2017_us_nation_5m.shp -where "GEOID NOT LIKE '%ZZZ'" -f GeoJSON ./data/sld-clipped.geojson ./data/sld.geojson
Exited with code 137
Hint: Exit code 137 typically means the process is killed because it was running out of memory
Hint: Check if you can optimize the memory usage in your app
Hint: Max memory usage of this container is 4287582208
I can take a hint ;)
https://circleci.com/gh/openstates/openstates-district-maps/12
Nice! My refactor to run the ogr2ogr
clipping on individual states instead of on the country as a whole has resulted in a pipeline that'll run within the 4 GB RAM max on CI.
https://circleci.com/gh/openstates/openstates-district-maps/13
Only remaining bit is to get CircleCI to pass the Mapbox environment variables to its "remote Docker environment." Otherwise, everything's running successfully:
https://circleci.com/gh/openstates/openstates-district-maps/26
Sort of a pain to do locally, since the job takes a long time and lots of DLs