search

temporalio / temporalite-archived

An experimental distribution of Temporal that runs as a single process
MIT License
610 stars 75 forks source link

Enable Advanced Visibility #23

Open gedw99 opened 2 years ago

gedw99 commented 2 years ago

https://github.com/prabhatsharma/zinc

100% golang built on top of the excellent Bluge golang code.

This will work well with SQLite for a single binary solution.

the http api matches Elastic too so easy to plug into . Not everything is supported yet

has a web gui. Not sure if temporalite has one yet .

no cli yet , but relatively easy to do with cobra and making http calls to server . Might already be an issue for it.

it’s fast too .

I am using it and it’s been great . It’s a new project and still got some rough edges compared to Elastic.

there is also the possibility to integrate via litestream but I doubt you need it. Litestream ( https://github.com/benbjohnson/litestream) gives you a type of changefeed off SQLite as well as real time backup to s3.

if you have any questions feel free to holler !

gedw99 commented 2 years ago

I just noticed that the sqlite driver is upstream. So maybe the elasticlite (zinc) can be done the exact same way - upstream it.

jlegrone commented 2 years ago

Very cool! I was not aware of the zinc project. Upstreaming support in temporalio/temporal would be my preference, as the storage interfaces tend to change a bit and this would make it much easier to stay up to date with the latest Temporal server releases.

We're also looking at litestream, but not sure yet if it needs to be a direct integration (ie. pulling it in as a library vs. running alongside Temporalite). Mentioned this here: https://youtu.be/Hz7ZZzafBoE?t=655 😄

gedw99 commented 2 years ago

hey @jlegrone

Very glad you are looking at litestream. it will be interesting if it can be HA'ed too. I spoke to Ben about the idea of doing CDC with litestream also.

Regarding Zinc. There is also a client implementation of elastic here: https://github.com/opensearch-project/opensearch-go The maintainer of Zinc knows about it...

jlegrone commented 2 years ago

I believe there are plans to add full visibility features (eg. filtering by custom attributes) to all sql storage drivers in temporalio/temporal. This should avoid the need for an elastic replacement in Temporalite, since the full visibility API will be supported natively by the SQLite storage driver.

We can keep this issue open to track enabling advanced visibility in Temporalite though.

Irvenae commented 1 year ago

+1 for this

karelbilek commented 1 year ago

Advanced visibility is already done in temporal 1.22 for sqlite...

karelbilek commented 1 year ago

I don't think this repo is maintained.

But if it is, you can merge #234 and that could enable advanced visibility