WeblateOrg / weblate

Web based localization tool with tight version control integration.
https://weblate.org/
GNU General Public License v3.0
4.63k stars 1.02k forks source link

Performance boost: Component Discovery Addon should not be available to repo links #2853

Closed fdosanjuan closed 5 years ago

fdosanjuan commented 5 years ago

Describe the bug Any time a VCS pull is made on a component, it triggers the component discovery process on the component linked to an actual repository and on all its linked childs, which makes the pull operation terribly slow.

To Reproduce Steps to reproduce the behavior:

  1. Configure a Git repository with a lot of files.
  2. Go to Weblate and create a project with one component linked to one of the files in the Git repo (let's name this component "master-component").
  3. Install the Component Discovery addon on master-component to match all the other files in the Git repo.
  4. It will take a while for Weblate to create all the components, but that's OK. Wait for it to finish.
  5. Modify one of the files in the Git repo (e.g., add a new term to translate)
  6. Back to Weblate, go to the repository management view of one of the components and perform a Pull operation. This will be very slow.

Expected behavior Once the pull button is pressed, the process to sync the repo should not be that time consuming.

The method Component::update_branch (in file trans/models/component.py) triggers a vcs_post_update on "master-component" and all of its linked childs. This eventually produces a call to post_update on each of the addons installed in the child component. And here lies the problem: since each linked child has the component discovery addon installed, it will be run.

I understand the need to run the component discovery on the "master-component", but not on its children.

Easy fix I have managed to temporarily fix it by modifying the method DiscoveryAddon::post_update (within file addons/discovery.py):

def post_update(self, component, previous_head):
        if not component.is_repo_link: 
                self.perform()

However, I am wondering whether a better solution would be to not install the Component Discovery addon automatically on child components (i.e., those created by Weblate as a result of the component discovery itself).

Notes on performance With ~350 child components, the previous fix makes the Pull operation run 7.5 times faster.

Server configuration and status

nijel commented 5 years ago

Thank you for your report, the issue you have reported has just been fixed.