matomo-org / plugin-EnvironmentVariables

Allows you to specify Matomo config in environment variables instead of the config file.
12 stars 14 forks source link

Mention in the docs that plugin can be enabled automatically by adding it to the config file and doesn't need to be enabled every time #10

Closed tomgi closed 3 days ago

tomgi commented 2 years ago

Hi, we've recently updated matomo from 4.5 to 4.8, and since then we've started having an issue with this plugin (or the specific way we're using this plugin).

We use this plugin to provide database credentials to matomo from environment variables, instead of having to hardcode them in config/config.ini.php.

One of the first things we do during deployment, before starting the matomo web server, is ./console plugin:activate EnvironmentVariables.

In the new version of matomo this started failing with the following error

./console plugin:activate EnvironmentVariables
ERROR [2022-03-17 23:45:12] 23  Uncaught exception: /var/www/html/libs/Zend/Db/Adapter/Pdo/Abstract.php(144): SQLSTATE[HY000] [2002] No such file or directory [Query: , CLI mode: 1]

  [Zend_Db_Adapter_Exception]                       
  SQLSTATE[HY000] [2002] No such file or directory  

  [PDOException]                                    
  SQLSTATE[HY000] [2002] No such file or directory  

plugin:activate [plugin1] ... [pluginN]

Seems like, in the new version of matomo, a working database connection is required for plugin activation.

Which creates a bit of a chicken and egg problem. EnvironmentVariables plugin is being used to tell matomo to get the database credentials from environment variables. But to be able to activate the plugin, matomo needs to have a working database connection already.

tsteur commented 2 years ago

To reproduce this issue, I removed the [database] section from my local config file.

Then I ran the same command and got this error:

$ ./console plugin:activate EnvironmentVariables
ERROR [2022-03-21 00:00:35] 99907  There was an error while updating the javascript tracker: Database access denied
ERROR [2022-03-21 00:00:35] 99907  There was an error while regenerating container releases: core/Db/Adapter.php(62): Database access denied [Query: , CLI mode: 1]
Activated plugin EnvironmentVariables
Error: error or warning logs detected, exit 1

The DB connection was required by

image

and also by the plugin manager (which was added a while back, maybe around Matomo 4 or so)

image

There may be also few other DB connections required.

@tomgi I would have assumed that this was already a problem with Matomo 4. And we had a similar issue reported back then with https://github.com/matomo-org/matomo/issues/16741

Is there any particular reason that you need to activate the environment variables plugin and can't have it persisted in the config file?

tomgi commented 2 years ago

Thanks for the investigation and reproduction @tsteur

On matomo 4.5, the environment variables plugin was able to get successfully activated without the DB credentials in the config file. We started having this issue only after we upgraded to matomo 4.8 recently.

The main reason we don't store DB credentials in the config file is that we use a single docker image to deploy matomo to multiple environments (staging and prod). We don't use permanent volumes to avoid configuration drift between environments. So in our setup, no files are persisted between deployments, and no files are shared between different containers that are deployed to the same environment.

We wouldn't want to store DB credentials in the docker image for security reasons. And that would also force us to have separate images per environment, which we would rather avoid.

As far as we understand it, and based on what we observed, in the previous version of matomo activating the environment variables plugin was injecting the DB credentials from ENV to the config file.

What we come up with as a temporary workaround, is to have empty strings as DB credentials in the config file, and automatically find-and-replace them from ENV as part of the deployment, before activating any plugins and before starting the matomo web server.

tsteur commented 2 years ago

thanks for this @tomgi

Sorry I wasn't quite clear in my comment. I meant would it be possible to maybe have the below line in your config file persisted, not the actual DB credentials:

Plugins[] = "EnvironmentVariables"

Within the Plugins[] section. Then the plugin be activated automatically.

tomgi commented 2 years ago

That's a great idea @tsteur! I didn't realise that adding the environment variables plugin to the config file is what it takes to activate it. I just tried it out and it works perfectly, thank you.

I think we can close this issue now.

If I might suggest, it might be worth adding a note to the readme of this plugin to mention that it can be safely activated this way (which I believe is not the general case for plugins that implement a custom activate function).

tsteur commented 2 years ago

Awesome, glad it works @tomgi I'll keep this issue open so we can document this

zymotik commented 2 years ago

...have the below line in your config file persisted, not the actual DB credentials:

Plugins[] = "EnvironmentVariables"

Within the Plugins[] section. Then the plugin be activated automatically.

Hi @tsteur, can this be done using the Matomo console? How would this be set? Thank you

tsteur commented 2 years ago

@zymotik you can edit the config/config.ini.php file directly. You can also use the console command to activate the plugin (./console plugin:activate EnvironmentVariables) but you'd want to make sure that this change in the config file will be persisted next time a new docker launches. Otherwise the entry will be lost again.

mbrodala commented 6 days ago

Since plugin:activate requires a working DB connection we have successfully pre-enabled this plugin in our Dockerfile like this:

RUN ./console config:set 'Plugins.Plugins[]="EnvironmentVariables"'
RUN ./console config:set 'PluginsInstalled.PluginsInstalled[]="EnvironmentVariables"'

Normally there is more to it but since this plugin does not even touch the DB (that's the point ;-) ), this should be enough for now.

(Tried with Matomo 5.1.2)