Unable to flush manifest from CLI

[dhensby]

At the moment manifests are built seperately depending on how (or who) you make requests against projects.

For example GET http://example.com/ will build a manifest inside the cache folder called "unknown".

using CLI and doing framework/sake / will build a manifest as your current CLI user.

This means it's not possible to flush frontend template caches via CLI. So automated deployments are a pain because template changes need to be flushed by an HTTP request.

[simonwelsh]

This is entirely a problem with deployment scripts running as a different user from the web user and the permissions problems related to it. There is nothing that SilverStripe can do here without making the manifest and other caches world writeable, which is a rather bad idea. Simply run your deploy scripts (or that part of it) as the same user that runs PHP.

[dhensby]

This doesn't solve the problem when SS creates a manifest as "unknown" which is what it does on all my systems. Running as the webserver will create a manifest for "apache" in that instance and not "unknown"

I have 3 manifests:

• apache (crons run as this)
• root (I run my commands as this coz I'm lazy, I could run it as my server user (dhensby))
• unkown (the manifest for the http requests)

[kinglozzer]

Happy to close this @dhensby?

[dhensby]

This is still a problem but I've worked around it because my deploy scripts now do rm -rf /path/to/webroot/silverstripe-cache/ && mkdir /path/to/webroot/silverstripe-cache/

However I think @sminnee was talking about removing the ability to put a cache folder in the webroot as it's gross. In that case deploy scripts would be a lot harder as the cache dir name changes and can be in many places (including in memory?!)

Whilst the computed temp dir continues to use the current executing user to generate it then we'll never be able to solve this problem without manual intervention in deploy scripts.

[tractorcow]

How about the ability to force-assign a temp dir via .env, so that you can set a temp location for your site regardless of user / php version?

[dhensby]

This actually a bigger problem on servers with systemd's privatetmp feature. Which makes trashing of Apaches tmp folders much harder.

[KINKCreative]

Like @simonwelsh already states, one only needs to run the script as the web user for it to be seamless and there's no changes needed in SS core.

sudo su - apache -c "framework/cli-script.php /dev/build flush=all"

Here are my deployer.php tasks if wanna just use them - confirming they work here:

task('silverstripe:build', function () {
    return run('sudo su - apache -c {{bin/php}} {{deploy_path}}/current/framework/cli-script.php /dev/build"');
})->desc('Run /dev/build');

task('silverstripe:buildflush', function () {
    return run('sudo su - apache -c "{{bin/php}} {{deploy_path}}/current/framework/cli-script.php /dev/build flush=all"');
})->desc('Run /dev/build?flush=all');

Just make sure that you replace apache if you have a different web user.

@kinglozzer seems like this is easily solvable now. And @dhensby, you should be able to set the user apache runs in httpd.conf, just make sure the user/group exists. This is definitely closeable.

[dhensby]

@KINKCreative I'm not sure it does solve the problem. On servers with privatetmp running you have to run as the system service to access the tmp dir for the apache process - being the user alone isn't enough.

[chillu]

On servers with privatetmp running you have to run as the system service to access the tmp dir for the apache process - being the user alone isn't enough.

Reading up on PrivateTmp, it seems like a feature on some linux distros (e.g. Fedora, RedHat). Does anyone know if its a default, and how wide spread the usage is?

Laravel allows writing to caches via CLI (e.g. php artisan cache:clear), and states that the webserver needs write access to that cache folder in https://laravel.com/docs/5.7. I think overall, it's a fairly well accepted practice in LAMP stacks to have caches warmed or changed via CLI tools, and then used or changed via web processes?

[sunnysideup]

how about:

wget https://site-set-in-env?flush=all&token=secret 

that is the purest way to do it I feel.

[dhensby]

Wowsers, what an old issue. I think I get round this problem now by executing the function as the webserver to rm -rf the cache dir.

Realistically this issue is not going to be fixed so I'm going to close it.

[sunnysideup]

Here is the solution https://github.com/sunnysideup/silverstripe-flush-front-end/

[michalkleiner]

The solution is to understand that the cache is sectioned/dependent on the user under which the request was run, and that it might be desirable to not flush from the CLI similarly to when a front-end request is made, so it's up to the devops folk to decide if it's needed and how to work with the system within its architecture and requirements, if it is needed.

I don't think this was an actual bug (seen through today's optics).

[sunnysideup]

@michalkleiner - agreed this is a not a bug, just an option to make Silverstripe more dev-friendly. Basically here is the user story

... as a developer, when I update a site (be it through automation (e.g. pipelines) or by hand) then I would like the front-end cache to be flushed also so that I do not have to remember to open the site and add ?flush=all to the url to ensure that, for example, the new database field that I added, is actually avaible for users.

I agree that there is a hypothetical situation where you want to flush the CLI cache but not the APACHE cache, but from my point of view that situation is rare (e.g. you can to test something on CLI first and have it randomly appear on the front-end when the fornt-end cache is flushed).

[michalkleiner]

Yep, I think we understand each other re: the overall picture.

The user story is good, that's the requirement. The solution to it is to engage a DevOps person or whoever is updating the site's code, and let them know about the specifics of the caching being dependent on the user running the process, so that they can factor that into the deployment process so that the said developer doesn't need to worry about it and their user story is catered for.

[sunnysideup]

some clients may not have enough $ to engage a devops professional to host a website.