search

GothenburgBitFactory / taskchampion

Personal task-tracking library
MIT License
80 stars 20 forks source link

Create an etebase-based backend #175

Closed djmitche closed 3 years ago

djmitche commented 3 years ago

Let's try supporting synchronizing to etebase. Details in #167.

djmitche commented 3 years ago

I was supposed to work on this during the week but my TaskWarrior sync broke again and it never made it onto the list.

savchenko commented 3 years ago

my TaskWarrior sync broke again

That's a sign :)

djmitche commented 3 years ago

I'm still struggling to figure out how to test this. I found https://hub.docker.com/r/grburst/etesync, which in theory should allow me to start a throwaway server for testing (so even though servers never actually delete anything, restarting that server between test runs will at least allow an easy cleanup).

But I can't even figure out how to "sign up" to that server, or honestly call anything -- I get 404's for the /api/v1/.. endpoints.

I'm also noticing that nothing has changed on any of the etesync repos since early March, and the main bit of work was in early January. I'm starting to have doubts :)

djmitche commented 3 years ago

I've gotten a suggestion to use https://hub.docker.com/r/victorrds/etesync instead

djmitche commented 3 years ago

..and look at etebase-go for CI examples

djmitche commented 3 years ago

So I got things started here, including figuring out how to set up a blank server to test against.

The major blocking issue I see is that there's no way to fetch an item by anything other than its uid, and its uid is server-generated. That is OK for the core functionality: we can add new history segments, get back a uid, and update that as the "latest' uid. It's a little tricky downloading history segments. Consider history H1 <- H2 <- H3 <- H4 -< H5, where the latest version on the replica is H2. On trying to add its new history segment with parent ID "H2', the replica is told "sorry, latest is H5". It must then iterate backward from H5 until it finds one with a parent of H2. For a replica that's been shut off for a long time, that could be a long process. The current implementation solves this by indexing the history segments by their parent ID, so finding the child of H2 is as easy as looking for a history segment with parent_id == "H2'.

This missing functionality is also problematic in that it means we can't switch from one server to another or fill in missing history on a crashed server (as uploading items to the new server would generate new uids). In particular, it makes a transfer from local to remote sync difficult.

I'm not sure if this is enough of a dealbreaker to throw this idea out entirely. In fact, I'm not even sure that this is a true observation about etesync, but I haven't gotten anyone to tell me I'm wrong.

savchenko commented 3 years ago

major blocking issue I see is that there's no way to fetch an item by anything other than its uid

Looking at the collections Python bindings, one should be able to fetch as:

  1. Whole collection (shall we call a particular task list a collection?), with the control of how many elements of the collection should be returned by the server.
  2. Changes since stoken was generated. This might be useful as we can store stoken on the client.
  3. Fetch in chunks.
  4. By UUID, as you have already mentioned.

There are also sections about transactions and using stoken. Based on these, do you reckon it's worth investing further time or TC can stay with its original server for the time being? No one knows the application better than you.

P.S. There is an impressive amount of users at #etebase / Freenode.

djmitche commented 3 years ago

Yes, the plan had been to store the entire task history in a single collection. It's also not possible to fetch collections by name, so revisiting that part of the plan doesn't help with this particular issue.

My instinct is that we shouldn't do this -- etesync is still early-stages (but then, so is taskchampion!) and I expect will have some bugs, breaking changes, and possibly issues with its own maintainership that might be a net drag on TC. If it did not force a change in the data model, as described above, I might be willing to include it as an option among others, but that does not appear to be the case.