splitgraph / seafowl

Analytical database for data-driven Web applications 🪶
https://seafowl.io
Apache License 2.0
392 stars 9 forks source link

GCS bucket support #379

Closed onpaws closed 1 year ago

onpaws commented 1 year ago
curl -i -H "Content-Type: application/json" \
-H "Authorization: Bearer redacted" \
-X POST "http://localhost:8080/q" -d@- <<EOF
{"query": "
CREATE TABLE holditright (id int)
"}
EOF
HTTP/1.1 200 OK
content-type: application/octet-stream
vary: Content-Type, Origin, X-Seafowl-Query
content-length: 0
date: Fri, 28 Apr 2023 05:58:58 GMT

Prototype seafowl.toml

[object_store]
type = "gcs"
bucket = "seafowl-gcsfuse"

[catalog]
type = "sqlite"
dsn = "./seafowl-data/seafowl.sqlite"

[frontend.http]
bind_host = "0.0.0.0"

TODO

neumark commented 1 year ago

LGTM 👍

onpaws commented 1 year ago

Was about to merge but at the last second I observed a pre-commit discrepancy between local (works) vs GHA CI ( observing this).

I tried running rustup update to pull in latest nightly, followed by cargo clippy --all-targets --workspace -- -D warnings which led me to observe this:

error: use of `default` to create a unit struct
    --> src/context.rs:1011:50
     |
1011 |             FileType::AVRO => Arc::new(AvroFormat::default()),
     |                                                  ^^^^^^^^^^^ help: remove this call to `default`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#default_constructed_unit_structs
     = note: `-D clippy::default-constructed-unit-structs` implied by `-D warnings`

error: could not compile `seafowl` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `seafowl` (lib test) due to previous error

So the intention of 9137aaa is an experiment to resolve that issue. Let's see 🍿

EDIT: looks like it passed

EDIT2: as did the rest of the CI build.

Therefore unless objections, I'm intending to merge. Please give me a shout if the AvroFormat change is a problem.

onpaws commented 1 year ago

Merging. Thanks for the review!