citizenos / citizenos-api

Citizen OS API application - https://api.citizenos.com/
Other
31 stars 8 forks source link

API: Topic content sync with Etherpad is slow and causes Etherpad performance to drop significantly #131

Closed tiblu closed 5 years ago

tiblu commented 5 years ago

Related issues:

tiblu commented 5 years ago

Why the sync?

  1. Loading Etherpad for every view is very slow, even in read-only mode, so when integrating the editor, we decided to load the Etherpad only when in "edit" mode to provide better user experience, specially with slow connections.
  2. There are views, where the full document is not shown, but parts of it - only the title for example. We need the updated title which is parsed from the Etherpad content on sync.
  3. Etherpad is a separate application with it's own database and server. We want to keep the loose coupling and not touch Etherpad internals from Citizen OS code directly.
  4. It's very important that when a vote is started, the latest version has been synced from EP and it never changes in the DB - everybody votes the same content.

The current sync

After code review , @ilmartyrk found that Citizen OS API does the sync with revision number, which causes the Etherpad (EP) API to fall into expensive revision fetch and apply logic. We're changing it to call it without revision, which falls into much cheaper logic on the Etherpad side.

Future of the sync?

tiblu commented 5 years ago

@ilmartyrk Discovered that our EP to Citizen OS DB sync does call to Etherpad API with revision number, which makes EP to fall into expensive logic. @ilmartyrk Changed it to use API without revision numbers, which is ALOT cheaper.

tiblu commented 5 years ago

Closing this issue, but we do need to have a discussion on the future - https://github.com/citizenos/citizenos-api/issues/131#issuecomment-481720616

KatiVellak commented 4 years ago

Legally reviewed, no impact.

loorm commented 4 years ago

Closing this issue, but we do need to have a discussion on the future - #131 (comment)

Apparently, this got fixed with Etherpad 1.8. So no longer an issue for the future. Issue fully closed and done.