search

zetkin / lyra

3 stars 3 forks source link

Fully sync cache with repo #25

Open richardolsson opened 11 months ago

richardolsson commented 11 months ago

image

WULCAN commented 8 months ago

On e5d32971b88e6576ce5f87db8df10ca9d0a8b564, translations are loaded from the local git clone on first use, and then never reloaded until the Lyra server process restarts.

The local git clone is updated from the remote before and after creating a pull-request so the translators' changes will be applied on a fresh copy, regardless of what the translator saw when she made the translations. The translator will never see this until the Lyra server process restarts, as the changes from the remote are never loaded into Lyra.

Try for example:

  1. Start Lyra targeting a project named p with a supported language code la and a base branch b
  2. Open /projects/p/la
  3. Make changes translation or the messages in the remote base_branch
  4. Reload /projects/p/la: you will not see the new changes
  5. Save some changes to the translations on /projects/p/la and create a pull request.
  6. Reload /projects/p/la: you will still not see the changes from the remote.

This is a problem because if we add new messages to a project, Lyra will have to be restarted to pick up those changes. When we restart Lyra, we risk losing translators' work that was made after the last pull request was created.

It is also a problem when we change parameters for a message. Translators will keep translating with the old parameters until Lyra is restarted.

It is also a problem when we change the default message of a message. Translators will keep translating the old, stale default message until Lyra is restarted.

Richard's diagram in this issue is a proposed solution that can resolve these problems. The diagram is already added to Lyra's repository, but the implementation does not match it yet.

WULCAN commented 2 months ago

There's a small set of HTTP operations like page loads and sever actions that need to interact with translation storage:

We imagine a single interface, to gate getting messages and translations (and likely to update them and to create pull requests) This could be Cache, Store or the data access layer.

This single interface will use an internal method Refresh as first step in some of its public methods: methods like getTranslations or accessProject. Refresh will pull from the remote, re-read translations from fs, and then merge the read translations into Store.

We will need a merge function that merges a previous state with a new state read from GitHub via git and a file system.

Then we add logic to skip reading from fs if pull did not find a new revision. For this we will need to add tracking of which revision translation storage was read from last.

Then we add logic to skip pull if we pulled recently. For this we will need to add tracking of when we last pulled.

In summary:

  1. Extract translation loading on translation page to dataAccess
  2. Implement Merge(old: StoreData, new StoreData): StoreData
  3. Implement a Refresh and use it in dataAccess
  4. Skip re-read and merge in Refresh if already based on HEAD
  5. Skip Pull in Refresh if too recent

We will want to move messages into Cache/Store too, as it is also read form remote, via fs, and also benefits from being refreshed. We do not need this before the steps above.

  1. Add messages caching into Store
  2. Use Cache or Store to get messages instead of directly with message adapter.

The server action for creating pull requests does not benefit from this new sync from remote. It just has to create a pull request from the state Lyra is already in. We can remove the current refresh-like implementation from it.

  1. Remove pull from sendPullRequest.
WULCAN commented 1 month ago

Before closing this issue, we need to resolve: