shp export fail

[td928]

in build run https://github.com/NYCPlanning/db-pluto/runs/6375507179?check_suite_focus=true#step:9:18 the shapefile export step failed

this is due to moving to github runner and installation for gdal is not explicit for the environment.

[td928]

after branch #331 successfully remove the error for failed postgis connection with github action. A new error appeared ERROR 6: Unknown option name '-nlt'

It should be noted one solution trying to solve above is by using a different docker gdal image that is perhaps more updated. But this does not remove the error and also would fail with the driver "FileGDB". It turns about to run "FileGDB" to create spatial file that the image needs additional SDK so gives us reason for using the current docker image.

Also it should be noted that the shapefile export function is actually quite different from the FGDB function so it is still failing after being updated to its new form. @SashaWeinstein

[SashaWeinstein]

Te can I have some more context on what you mean by "shapefile export function is actually quite different from the FGDB function"?

[td928]

yeah good question. I was a little naive when I updated the shape export function. The new version actually needs a $geomtype argument. I think it might be why it still failed this time. It is also different in that they are pointing to different tables in build engine to export e.g. mapluto vs. mapluto_gdb

[SashaWeinstein]

Ah ok you're talking about the specific implementations of the SHP_export and FGDB_export in the config.sh?

[td928]

that's right

[SashaWeinstein]

Ah ok so we need to do something more than borrow the SHP_export code from COLP?

[td928]

testing first hypothesis right now. Add geomtype argument. Otherwise remove -nlt flag might be second option

[SashaWeinstein]

I can report back on implications of running without -nlt flag if it ever finishes on my machine lol

[td928]

lol yeah takes surprising long time

[SashaWeinstein]

Te I have one insights and one question for you

The insight is that it seems clear that the webmapp/osgdal-docker imagine is not sufficient. I get this error ERROR: column s.consrc does not exist which google says is due to an out of date version of gdal. If the osgeo/gdal image isn't fit for purpose maybe there is another that we could use?

The question is why there are two docker runs in the FGDB_export function. I can't figure out the difference, or why we would need two

[SashaWeinstein]

Trying with osgeo/gdal just to see what happens

[td928]

so passing an additional geomtype argument for new shp export still failed :( how is the outcome for removing the -nlt flag for you @SashaWeinstein? Interesting the error you saw was something I haven't seen. I have tried the osgeo/gdal and it threw an error for FileGDB driver see my comments above and also does not solve the shapefile issue.

[td928]

also why does local exporting takes such a long time. According to another successful build couple months ago only took about 10 minutes in exporting step. What's the difference when doing this locally?

[SashaWeinstein]

I'm not making much progress, I've tried several images and the webmapp image is the only image I can find that produces any output.

No clue on what determines the time it takes to export

[td928]

ok. I do have an answer for the two exporting step for FGDB. One is for the mapped and one is for the unmapped.