Support for Database _changes

[adeelsohailahmed]

Hello,

Are there any plans to support CouchDB's /db/_changes API?

[bmario]

There are no concrete plans, but I looked a bit into it and have the feeling using the continuous mode in a loop would be the best way to implement this. I imagine the usage somewhere near this:

@dataclass
class DeletedEvent:
  doc: str
  rev: str

  async def doc(self) -> Document:
    return await database[doc]

@dataclass
class ChangeEvent:
  doc: str
  new_rev: str
  old_rev: str

  async def doc(self) -> Document:
    return await database[doc]

# loops forever and for every received event, there's one iteration
async for event in database.changes(args):
  if isinstance(event, DeleteEvent):
    print(f"The document {event.doc} was deleted.")
  elif isinstance(event, ChangeEvent):
    print(f"The document {event.doc} was updated from {event.old_rev} to {event.new_rev}")

I'm not sure if the include_docs parameter should be used, though, or if aiocouch should do the download of the new revision only on demand...

[adeelsohailahmed]

Using the iterator for database changes certainly seems to be the best way to approach this. This is exactly what python-cloudant has done as well.

All three options (poll or normal, longpoll, and continuous) should be supported, if possible.

include_docs parameter shouldn't be used by default, in my opinion. It should be used only when user specifies it themselves. (It could be a bool that defaults to False. This way, only those users who do need full contents of each document change, can retrieve it).

For the implementation, I've been thinking that we could have a Feed class in feed.py (similar to python-cloudant). It's not meant to be used directly, but via database.changes() instead (like you suggested in the example above).

Internally, Feed class itself would use a Remote class. We could either extend RemoteDatabase to have _changes method, or we could create another class in remote.py.

[bmario]

I worked on a prototype. It's not remotely complete yet, but could this work in your use case?

https://github.com/metricq/aiocouch/tree/experimental-changes-endpoint

[adeelsohailahmed]

My current use case is very basic for now. I'm just using normal poll every now and then to check if there were any changes.

Some comments on the prototype:

• feed option should not be forced to continuous (it should default to normal poll unless explicitly stated otherwise). We could have another method db.infinite_changes() that always uses a continuous feed.
• since option should default to 0 (instead of defaulting to now). As a user, I expect to get all changes if I don't specify a since explicitly. This expectation is also consistent with CouchDB's _changes API default options.
• last_seq should be supported. This should be returned after the iterator has exhausted (in case of normal polling).
• include_docs should be supported as well. This way, I don't have to make a separate request to get the changed data.

PS: Not sure if this is intended, but the change_events example in its current state leads to an infinite loop. To trigger it, you just have make one change in the database manually, and then it goes bonkers (adds a random UUID to fluffy in the changed doc, saves it remotely, gets this change from CouchDB, and repeats the process).

[adeelsohailahmed]

Hey @bmario, sorry to be a bother, but what's the update on this?

[bmario]

Well, before my vacation I updated the branch again to levitate some of your concerns. Had a look at the latest commits?

[adeelsohailahmed]

I just had a look at the latest commits. Works great for me, thank you! Would love to see this branch included in the official release!

[adeelsohailahmed]

Hi @bmario, any plans to get the experimental _changes branch merged with the master? It's been a while...