Open stuartc opened 2 weeks ago
@stuartc
I guess we'll use use Authorization header like BEARER <token>
for all requests? Probably base64 encoded. Run tokens aren't sensitive, but when we add client API keys they will be.
[EDIT: pending our conversation in slack maybe it's /collections/<collection_name>?key=xxx
I think GET needs to be like this:
GET /collections/<collection_name>/<key-pattern>
Where key-pattern is optional
With query parameters (now or in future): offset
, limit
, since/after
?, until/before
?
I think the POST is always an upsert (if if a record exists, it'll be removed)
I have an idea that the post body is an array of the form { key: string, record: any, meta: any }
. Meta would allow us to do things like override timestamps (maybe), or set expiration (perhaps?) per record. I don't know if we need it but we could use that as a bucket of system stuff.
What, if anything, should post return? An array with a status for each item? Maybe something more like { upserted: 101, errors: [] }
We also need a DELETE /collections/<collection_name>/<key>
. Should we support a key pattern here or is that too exciting?
Updated the main issue body for GET/POST/DELETE requests, as per the Canvas conversation
Re: https://github.com/OpenFn/lightning/issues/2190
Provide an HTTP endpoint that exposes methods that calls functions inside the Collection context.
Streaming is a requirement of this feature, we should be using features like
send_chunked
andchunk
(from Plug).Auth
Access to this API must be authenticated using either a Users access token, or a RunToken, in the Authentication Header
API spec
GET /collections
Fetches/searches for values within a collection
/collections/<collection_name>
returns everything in the collection*
for pattern matchingupdated_after
/created_after
only returns records GTE this the start of (last second? of) this dateupdated_before
/created_before
only returns records LTE the end of (last second? of) this datelimit
only return the first N recordsoffset
search from this offset in the dbmeta
flag to include metadata (timestamps) in as a_meta
key on each valueReturns:
Each item is a
{ key , value, created, updated }
object (I'd like a name for this wrapper - Record?)Other metadata, like
end_offset
for paged results, is included at the top of the object as siblings ofresults
POST /collections
Upsert values into a collection
{ key, value, ...meta }
objects (where meta is not used now but would be a collection of metadata properties)Returns
For any 20x payload, return a JSON summary in the form:
DELETE /collections
Remove values from a collection
*
for pattern matchingupdated_after
/created_after
only returns records GTE this the start of (last second? of) this dateupdated_before
/created_before
only returns records LTE the end of (last second? of) this dateReturns
If possible, return a JSON summary in the form: