Plugins installation improvements

SchrodingersGat commented 5 days ago

Replaces https://github.com/inventree/InvenTree/pull/8502
Closes https://github.com/inventree/InvenTree/issues/7709

Summary

This PR improves plugin loading and install to ensure that the plugins are properly synced between different running containers, e.g. when running under docker. Ensures that the required system packages are installed before collecting plugins.

Installation Process

Install plugin via the web interface (frontend server)
Plugin is installed and added to plugins.txt
Activate plugin (via frontend server)
Server offloads task (to background worker) to copy static files, etc
Background worker checks plugins.txt - sees it has changed - and installs the new plugin
Background worker then runs offline processing for the newly activated plugin

netlify[bot] commented 5 days ago

Deploy Preview for inventree-web-pui-preview ready!

Name	Link
Latest commit	566fea5711ffa15f33f980fd97df92347c084b68
Latest deploy log	https://app.netlify.com/sites/inventree-web-pui-preview/deploys/673d11a67a6e02000858458b
Deploy Preview	https://deploy-preview-8503--inventree-web-pui-preview.netlify.app
Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 100 (no change from production) Accessibility: 86 (no change from production) Best Practices: 100 (no change from production) SEO: 78 (no change from production) PWA: - View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

codecov[bot] commented 5 days ago

Codecov Report

Attention: Patch coverage is 75.29412% with 21 lines in your changes missing coverage. Please review.

Project coverage is 84.45%. Comparing base (3a81e03) to head (566fea5). Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
src/backend/InvenTree/plugin/installer.py	71.15%	15 Missing :warning:
src/backend/InvenTree/plugin/helpers.py	57.14%	6 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #8503 +/- ## ========================================== - Coverage 84.61% 84.45% -0.17% ========================================== Files 1178 1179 +1 Lines 53582 53693 +111 Branches 2026 2029 +3 ========================================== + Hits 45341 45349 +8 - Misses 7718 7823 +105 + Partials 523 521 -2 ``` | [Flag](https://app.codecov.io/gh/inventree/InvenTree/pull/8503/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=inventree) | Coverage Δ | | |---|---|---| | [backend](https://app.codecov.io/gh/inventree/InvenTree/pull/8503/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=inventree) | `85.94% <75.29%> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=inventree#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

SchrodingersGat commented 4 days ago

@wolflu05 I have refactored this (again) - a much simpler approach now which ensures that the plugins get installed into all running containers as appropriate

wolflu05 commented 3 days ago

Thanks, that looks promising, I'd like to take a closer look and test it myself, but that can take me some days...

SchrodingersGat commented 3 days ago

Let me know when you've had a chance to test it. This is one of the final outstanding Issues for 0.17.0

SchrodingersGat commented 3 days ago

@matmair if you have a chance to cast your eye over this I'd appreciate it :)

matmair commented 2 days ago

@SchrodingersGat I might be blind but where exactly is the offloading happening? Maybe I am too tired but I seem to miss that change in the diff

SchrodingersGat commented 2 days ago

@SchrodingersGat I might be blind but where exactly is the offloading happening? Maybe I am too tired but I seem to miss that change in the diff

When you activate a plugin, the collection of static files is offloaded to the background worker:

For this to succeed, the background worker needs to have the same set of plugins installed!

matmair commented 1 day ago

Ah now I found the link, the check_for_migrations function is doing the installing

wolflu05 commented 1 day ago

Sorry, I had no time yet, I was planning to test this on the weekend. But if anything pops up then, I'll just reply here.

SchrodingersGat commented 1 day ago

Sorry @wolflu05 got my wires crossed. LMK if you find anything

inventree / InvenTree