search

protomaps / go-pmtiles

Single-file executable tool for working with PMTiles archives
BSD 3-Clause "New" or "Revised" License
356 stars 49 forks source link

Move "main.go" in to "app/pmtiles/pmtiles.go" to allow custom gocloud.dev/blob imports #91

Closed thisisaaronland closed 11 months ago

thisisaaronland commented 11 months ago

This PR moves the guts of main.go in to a Run method in app/pmtiles/pmtiles.go so that this code can be used with bespoke and custom blob.Bucket implementations. The original main.go file remains and has been updated to call app/pmtiles.Run().

As a concrete example, I generally access S3 buckets using the aaronland/gocloud-blob-s3 package which is a thin wrapper to create AWS session instances using named credentials strings not otherwise supported by the Go AWS packages. For example:

go run -mod readonly cmd/pmtiles/main.go show \
    example.pmtiles \
    --bucket='s3blob://BUCKET?prefix=PREFIX&region=REGION&credentials=session'

Specifically s3blob://BUCKET?prefix=point-in-polygon/&region=REGION&credentials=session

Where go-sfomuseum-pmtiles/cmd/pmtiles/main.go looks like this:

https://github.com/sfomuseum/go-sfomuseum-pmtiles/blob/app/cmd/pmtiles/main.go

I live for the day when I can retire gocloud-blob-s3 but that is still not possible yet. The point of the PR is not to support that package specifically so much as custom blob.Bucket providers that people may want to use but that shouldn't be bundled in protomaps/go-pmtiles by default.

bdon commented 11 months ago

This seems like a big change to work around a method of providing credentials. Can you find another way to do this? Like providing an implementation of the pmtiles Bucket with your custom blob module?

thisisaaronland commented 11 months ago

The issue would still be the same: Once compiled the pmtiles command will have no knowledge of the custom implementation (be it a pmtiles.Bucket or a blob.Bucket) because it will not have been imported in main.go.

All the PR does is move the core of the logic in main.go in to a package that can be imported by external programs, which in turn can load custom or bespoke logic around buckets.

That said, looking at the PR above there do appear to be way more changes than what I've just described. I will investigate.

bdon commented 11 months ago

I would really prefer that you copy the necessary code from main.go into your application. I don't envision a general need for reusing the code in main.go - it should call what is eventually the "programmatic" API in the pmtiles/ module. Otherwise it becomes my job to create and maintain an entire 2nd API surface that is the logic in main.go.

thisisaaronland commented 11 months ago

Fair enough.