Plugin installation fail

Summary

Steps to reproduce

I have this problem since mattermost 7.7 and now i'm using 7.10.2 and the problem is still here. I use the container team-edition version deployed via the chart helm https://github.com/mattermost/mattermost-helm/tree/master/charts/mattermost-team-edition on openshift container plantform.

Posts on the forum releted to same problem but without any solution : https://forum.mattermost.com/t/plugin-activation-problem/14865 https://forum.mattermost.com/t/mattermost-v7-9-1-team-edition-helm-deployment-issues/15434/18

Expected behavior

When mattermost start, pre packaged plugins are installed

Observed behavior (that appears unintentional)

Plugins installation fail

{"timestamp":"2023-06-26 07:11:30.844 Z","level":"info","msg":"Server is initializing...","caller":"platform/service.go:165","go_version":"go1.19.5"} {"timestamp":"2023-06-26 07:11:30.844 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:242","database":"master","dataSource":"****:****@tcp(*******:3306)/mattermost?tls=skip-verify&writeTimeout=30s&charset=utf8mb4%2Cutf8"} {"timestamp":"2023-06-26 07:11:30.921 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:242","database":"master","dataSource":"****:****@tcp(**********:3306)/mattermost?multiStatements=true&tls=skip-verify&writeTimeout=30s&charset=utf8mb4%2Cutf8"} {"timestamp":"2023-06-26 07:11:31.014 Z","level":"debug","msg":"Deleting any unused pre-release features","caller":"sqlstore/preference_store.go:25"} {"timestamp":"2023-06-26 07:11:31.026 Z","level":"info","msg":"Starting websocket hubs","caller":"platform/web_hub.go:95","number_of_hubs":16} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":1} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":3} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":5} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":2} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":9} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":11} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":6} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":12} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":7} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":8} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":13} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":10} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":14} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":15} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":4} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"debug","msg":"Hub is starting","caller":"platform/web_hub.go:373","index":0} {"timestamp":"2023-06-26 07:11:31.027 Z","level":"info","msg":"Loaded system translations","caller":"i18n/i18n.go:132","for locale":"fr","from locale":"/mattermost/i18n/fr.json"} {"timestamp":"2023-06-26 07:11:31.031 Z","level":"info","msg":"Ensuring the telemetry ID","caller":"telemetry/telemetry.go:142","id":"4en7kzzozbd43xswnkaaq9oquy"} {"timestamp":"2023-06-26 07:11:31.049 Z","level":"info","msg":"Current version is 7.10.2 (7.10.2/Thu May 18 15:30:39 UTC 2023/42f472f34b27f3dd0fb7df266fef903673331175/none)","caller":"app/server.go:412","current_version":"7.10.2","build_number":"7.10.2","build_date":"Thu May 18 15:30:39 UTC 2023","build_hash":"42f472f34b27f3dd0fb7df266fef903673331175","build_hash_enterprise":"none"} {"timestamp":"2023-06-26 07:11:31.049 Z","level":"info","msg":"Team Edition Build","caller":"app/server.go:423","enterprise_build":false} {"timestamp":"2023-06-26 07:11:31.049 Z","level":"info","msg":"Printing current working","caller":"app/server.go:427","directory":"/mattermost"} {"timestamp":"2023-06-26 07:11:31.049 Z","level":"info","msg":"Loaded config","caller":"app/server.go:428","source":"****:****@tcp(***********:3306)/mattermost?readTimeout=30s&tls=skip-verify&writeTimeout=30s&charset=utf8mb4%2Cutf8"} {"timestamp":"2023-06-26 07:11:31.050 Z","level":"debug","msg":"Will fetch notices from","caller":"app/product_notices.go:344","url":"https://notices.mattermost.com/","skip_cache":false} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"info","msg":"Starting workers","caller":"jobs/workers.go:48"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ExportDelete"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ProductNotices"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"InstallNotifyAdmin"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"HostedPurchaseScreening"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker Started","caller":"indexer/indexing_job.go:100","workername":"BleveIndexer"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ExportProcess"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"migrations/worker.go:50","worker":"Migrations"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ExtractContent"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Watcher Started","caller":"jobs/jobs_watcher.go:38"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"scheduler/worker.go:39","worker":"Plugins"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ImportProcess"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"info","msg":"Starting schedulers.","caller":"jobs/schedulers.go:47"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Cleaning up token store.","caller":"app/server.go:1238"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Cleaning up command webhook store.","caller":"sqlstore/command_webhook_store.go:93"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Cleaning up session store.","caller":"app/server.go:1253"} {"timestamp":"2023-06-26 07:11:31.072 Z","level":"debug","msg":"Worker started","caller":"jobs/base_workers.go:37","worker":"ImportDelete"} {"timestamp":"2023-06-26 07:11:31.093 Z","level":"debug","msg":"We could not find the license key in the database or on disk at","caller":"utils/license.go:104","filename":"/mattermost/config/mattermost.mattermost-license"} {"timestamp":"2023-06-26 07:11:31.100 Z","level":"error","msg":"License key from https://mattermost.com required to unlock enterprise features.","caller":"platform/license.go:101","error":"resource: License id: "} {"timestamp":"2023-06-26 07:11:31.101 Z","level":"debug","msg":"Skipping license expiration check because no license is expected on Team Edition","caller":"app/server.go:1355"} {"timestamp":"2023-06-26 07:11:31.102 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"product_notices","next_runtime":"2023-06-26 08:11:31.102603158 +0000 UTC m=+3600.683037398"} {"timestamp":"2023-06-26 07:11:31.127 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"import_delete","next_runtime":"2023-06-27 07:11:33.007017325 +0000 UTC m=+86402.587451555"} {"timestamp":"2023-06-26 07:11:31.137 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"export_delete","next_runtime":"2023-06-27 07:11:32.159558105 +0000 UTC m=+86401.739992336"} {"timestamp":"2023-06-26 07:11:31.137 Z","level":"debug","msg":"Scheduler: isEnabled: false","caller":"notify_admin/scheduler.go:20","scheduler":"upgrade_notify_admin"} {"timestamp":"2023-06-26 07:11:31.137 Z","level":"debug","msg":"Scheduler: isEnabled: false","caller":"notify_admin/scheduler.go:20","scheduler":"trial_notify_admin"} {"timestamp":"2023-06-26 07:11:31.137 Z","level":"debug","msg":"Scheduler: isEnabled: true","caller":"notify_admin/install_plugin_scheduler.go:20","scheduler":"install_plugin_notify_admin"} {"timestamp":"2023-06-26 07:11:31.153 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"install_plugin_notify_admin","next_runtime":"2023-06-27 07:11:32.989136331 +0000 UTC m=+86402.569570561"} {"timestamp":"2023-06-26 07:11:31.157 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"migrations","next_runtime":"2023-06-26 07:12:31.157463737 +0000 UTC m=+60.737897971"} {"timestamp":"2023-06-26 07:11:31.164 Z","level":"debug","msg":"Next run time for scheduler","caller":"jobs/schedulers.go:147","scheduler_name":"plugins","next_runtime":"2023-06-27 07:11:31.758905735 +0000 UTC m=+86401.339339965"} {"timestamp":"2023-06-26 07:11:31.164 Z","level":"debug","msg":"Scheduler: isEnabled: false","caller":"last_accessible_post/scheduler.go:20","scheduler":"last_accessible_post"} {"timestamp":"2023-06-26 07:11:31.164 Z","level":"debug","msg":"Scheduler: isEnabled: false","caller":"last_accessible_file/scheduler.go:20","scheduler":"last_accessible_file"} {"timestamp":"2023-06-26 07:11:31.234 Z","level":"debug","msg":"Initializing web routes","caller":"web/web.go:26"} {"timestamp":"2023-06-26 07:11:31.234 Z","level":"debug","msg":"Rewriting static assets","caller":"utils/subpath.go:83","from_subpath":"/","to_subpath":"/"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"error","msg":"Failed to update assets subpath from config","caller":"web/static.go:30","error":"failed to update root.html with subpath /: open /mattermost/client/root.html: permission denied"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"debug","msg":"Using client directory","caller":"web/static.go:34","clientDir":"/mattermost/client"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"info","msg":"Starting up plugins","caller":"app/plugin.go:218"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"debug","msg":"Enabling plugin health check job","caller":"plugin/environment.go:613","interval_s":"30s"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"info","msg":"Syncing plugins from the file store","caller":"app/plugin.go:299"} {"timestamp":"2023-06-26 07:11:31.235 Z","level":"debug","msg":"Plugin health check job starting.","caller":"plugin/health_check.go:31"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-custom-attributes-v1.3.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-jitsi-v2.0.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-nps-v1.3.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-gitlab-v1.6.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-aws-SNS-v1.2.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-welcomebot-v1.2.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-confluence-v1.3.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.237 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-calls-v0.15.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.242 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-todo-v0.6.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.284 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-jenkins-v1.1.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.285 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-antivirus-v0.1.2-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.285 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/focalboard-v7.10.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.384 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-jira-v3.2.2-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.384 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-apps-v1.2.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.385 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-autolink-v1.2.2-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.484 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-zoom-v1.6.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.588 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-github-v2.1.5-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.685 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-playbooks-v1.36.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:31.785 Z","level":"debug","msg":"Processing prepackaged plugin","caller":"app/plugin.go:984","path":"/mattermost/prepackaged_plugins/mattermost-plugin-channel-export-v1.0.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:34.497 Z","level":"debug","msg":"Installing prepackaged plugin","caller":"app/plugin.go:1014","path":"/mattermost/prepackaged_plugins/mattermost-plugin-nps-v1.3.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:34.599 Z","level":"error","msg":"Failed to install prepackaged plugin","caller":"app/plugin.go:964","path":"/mattermost/prepackaged_plugins/mattermost-plugin-nps-v1.3.1-linux-amd64.tar.gz","error":"Failed to install extracted prepackaged plugin /mattermost/prepackaged_plugins/mattermost-plugin-nps-v1.3.1-linux-amd64.tar.gz: installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., mkdir plugins/com.mattermost.nps: permission denied"} {"timestamp":"2023-06-26 07:11:35.506 Z","level":"debug","msg":"Installing prepackaged plugin","caller":"app/plugin.go:1014","path":"/mattermost/prepackaged_plugins/mattermost-plugin-apps-v1.2.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:35.587 Z","level":"error","msg":"Failed to install prepackaged plugin","caller":"app/plugin.go:964","path":"/mattermost/prepackaged_plugins/mattermost-plugin-apps-v1.2.1-linux-amd64.tar.gz","error":"Failed to install extracted prepackaged plugin /mattermost/prepackaged_plugins/mattermost-plugin-apps-v1.2.1-linux-amd64.tar.gz: installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., mkdir plugins/com.mattermost.apps: permission denied"} {"timestamp":"2023-06-26 07:11:35.596 Z","level":"debug","msg":"Installing prepackaged plugin","caller":"app/plugin.go:1014","path":"/mattermost/prepackaged_plugins/mattermost-plugin-calls-v0.15.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:35.603 Z","level":"error","msg":"Failed to install prepackaged plugin","caller":"app/plugin.go:964","path":"/mattermost/prepackaged_plugins/mattermost-plugin-calls-v0.15.1-linux-amd64.tar.gz","error":"Failed to install extracted prepackaged plugin /mattermost/prepackaged_plugins/mattermost-plugin-calls-v0.15.1-linux-amd64.tar.gz: installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., mkdir plugins/com.mattermost.calls: permission denied"} {"timestamp":"2023-06-26 07:11:35.731 Z","level":"debug","msg":"Installing prepackaged plugin","caller":"app/plugin.go:1014","path":"/mattermost/prepackaged_plugins/mattermost-plugin-playbooks-v1.36.1-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:35.789 Z","level":"error","msg":"Failed to install prepackaged plugin","caller":"app/plugin.go:964","path":"/mattermost/prepackaged_plugins/mattermost-plugin-playbooks-v1.36.1-linux-amd64.tar.gz","error":"Failed to install extracted prepackaged plugin /mattermost/prepackaged_plugins/mattermost-plugin-playbooks-v1.36.1-linux-amd64.tar.gz: installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., mkdir plugins/playbooks: permission denied"} {"timestamp":"2023-06-26 07:11:35.935 Z","level":"debug","msg":"Installing prepackaged plugin","caller":"app/plugin.go:1014","path":"/mattermost/prepackaged_plugins/focalboard-v7.10.0-linux-amd64.tar.gz"} {"timestamp":"2023-06-26 07:11:35.944 Z","level":"error","msg":"Failed to install prepackaged plugin","caller":"app/plugin.go:964","path":"/mattermost/prepackaged_plugins/focalboard-v7.10.0-linux-amd64.tar.gz","error":"Failed to install extracted prepackaged plugin /mattermost/prepackaged_plugins/focalboard-v7.10.0-linux-amd64.tar.gz: installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., mkdir plugins/focalboard: permission denied"} {"timestamp":"2023-06-26 07:11:35.967 Z","level":"debug","msg":"Able to write files to local storage.","caller":"filestore/localstore.go:77"} {"timestamp":"2023-06-26 07:11:35.985 Z","level":"info","msg":"Starting Server...","caller":"app/server.go:899"} {"timestamp":"2023-06-26 07:11:35.985 Z","level":"info","msg":"Server is listening on [::]:8065","caller":"app/server.go:971","address":"[::]:8065"} {"timestamp":"2023-06-26 07:11:35.985 Z","level":"debug","msg":"No license provided; Remote Cluster services disabled","caller":"app/server.go:579"} {"timestamp":"2023-06-26 07:11:43.390 Z","level":"info","msg":"Failed to fetch marketplace plugins for telemetry. Using predefined list.","caller":"telemetry/telemetry.go:1443","error":"Get \"https://api.integrations.mattermost.com/api/v1/plugins?build_enterprise_ready=false&cloud=false&enterprise_plugins=false&filter=&local_only=false&page=0&platform=&plugin_id=&remote_only=false&return_all_versions=false&server_version=7.10.2\": dial tcp 52.222.169.100:443: i/o timeout"}

Possible fixes

It looks like the user that run the container do not have enough right

mattermost / mattermost