wavded / ogr2ogr

An ogr2ogr wrapper library
MIT License
216 stars 46 forks source link

Possible to run serverless AWS Lambda ogr2ogr function? #45

Closed stdmn closed 4 years ago

stdmn commented 5 years ago

I've been trying to run a AWS Lambda function that allows me to use ogr2ogr to post to an AWS RDS Postgres DB and am running up against some problems. Body being passed is a simple geoJSON for testing purposes.

I'm guessing it has to do with this: ogr2ogr requires the command line tool ogr2ogr

Any hack-y way that I can get it running on a Lambda server? Example express "POST" function below:

app.post("/items", async function(req, res) {
    try {
        const ogr = ogr2ogr(req.body)
            .format("PostgreSQL")
            .destination(
                "PG:dbname=<dbname> user=<user> port=5432 password=<password> host=<host>"
            )
            .options(["-nln", "test"])
            .timeout(30000)
            .promise();

        ogr.resolve(resolve => {
            res.json({ success: "post call succeed!", url: req.url, body: req.body });
        });
    } catch (err) {
        console.log(err);
        res.json({ failure: err });
    }
});

Lambda log:

- 19:11:40 START Version: $LATEST
- 19:11:40 App started
- 19:11:40 EVENT: {"resource":"/items","path":"/items","httpMethod":"POST","headers":{"Accept":"application/json, text/plain, */*","Accept-Encoding":"gzip, deflate, br","Accept-Language":"en-US,en;q=0.9","CloudFront-Forwarded-Proto":"https","CloudFront-Is-Desktop-Viewer":"true","CloudFront-Is-Mobile-Viewer":"false","CloudFront-Is-SmartTV-Viewer":"
- 19:11:41 Error: Error: spawn ogr2ogr ENOENT at ChildProcess.<anonymous> (/var/task/node_modules/ogr2ogr/index.js:222:18) at emitTwo (events.js:126:13) at ChildProcess.emit (events.js:214:7) at maybeClose (internal/child_process.js:925:16) at Socket.stream.socket.on (internal/child_process.js:346:11) at emitOne (events.js:
- 2019-06-25T19:11:41.150Z: 
- Error: Error: spawn ogr2ogr ENOENT
- at ChildProcess.<anonymous> (/var/task/node_modules/ogr2ogr/index.js:222:18)
- at emitTwo (events.js:126:13)
- at ChildProcess.emit (events.js:214:7)
- at maybeClose (internal/child_process.js:925:16)
- at Socket.stream.socket.on (internal/child_process.js:346:11)
- at emitOne (events.js:116:13)
- at Socket.emit (events.js:211:7)
- at Pipe._handle.close [as _onclose] (net.js:567:12)
- 19:11:41: END 
- 19:11:41: REPORT Duration: 771.88 ms  Billed Duration: 800 ms Memory Size: 128 MB Max Memory Used: 76 MB

One additional note: I also am using the GDAL Lambda layer from Geolambda, but not sure if there's anything else I need to do to add other than connecting to the layer to the function

wavded commented 5 years ago

I haven't tried this at all so I'm not sure, the errors indicate that you don't have the gdal libraries installed on the system or container.

wavded commented 4 years ago

Closing due to age, reopen if issue persists.

ferily7 commented 3 years ago

@wavded I am actually trying to do this same thing as well. Do you know how I can install GDAL on an AWS Lambda function?

I tried to do the .command('docker run -v /home/:/home --rm osgeo/gdal:alpine-small-latest ogr2ogr') but it didn't work for me.

@stdmn did you manage to figure out how to run ogr2ogr on a lambda?

wavded commented 3 years ago

I am unfamiliar with AWS Lambda unfortunately. However, this could make a great wiki entry if you guys figure it out for anybody wanting to do this in the future.

ferily7 commented 3 years ago

Sure, we can do that. I'm currently trying to figure out how to run the ogr2ogr function on a AWS Lambda. Actually quick question for you @wavded

When I tried to run the ogr2ogr function in my code, I get a Error: ogr2ogr: error while loading shared libraries: libpcre.so.0: cannot open shared object file: No such file or directory -- not sure what that means?

wavded commented 3 years ago

Haven't seen that error, seems like the environment is missing some libraries or a library isn't linked properly. May have luck googling for that sort of thing.