magento / magento2

Prior to making any Submission(s), you must sign an Adobe Contributor License Agreement, available here at: https://opensource.adobe.com/cla.html. All Submissions you make to Adobe Inc. and its affiliates, assigns and subsidiaries (collectively “Adobe”) are subject to the terms of the Adobe Contributor License Agreement.
http://www.magento.com
Open Software License 3.0
11.59k stars 9.32k forks source link

Merged CSS/JS files require pub/static/_cache writeable, contrary to documentation #13225

Open scottsb opened 6 years ago

scottsb commented 6 years ago

Preconditions

  1. Have Magento installed with two filesystem owners as documented, with all code files (source and CLI-generated) read-only for web server user. This includes the full pub/static directory.
  2. Turn on the merging of CSS or JS files.

Steps to reproduce

Clear static file cache and load home page.

Expected result

No exception should be generated.

Actual result

Exception is logged:

[2018-01-16 16:18:56] main.CRITICAL: Directory "/var/www/html/___/pub/static/_cache/merged" cannot be created Warning!mkdir(): Permission denied {"exception":"[object] (Magento\\Framework\\Exception\\FileSystemException(code: 0): Directory \"/var/www/html/___/pub/static/_cache/merged\" cannot be created Warning!mkdir(): Permission denied at /var/www/html/___/vendor/magento/framework/Filesystem/Driver/File.php:225)"} []

Commentary

This could arguably be seen as a documentation issue in which case I can report it in the devdocs repo. However, as I understand it, it seems that the documentation does describe the intended state, meaning the code should be changed instead. I'm not sure if it's possible, but if it were, ideally the merged files would be generated by static-content:deploy as alluded to in #9542. This would allow multiple servers in a cluster to be certain to have the merged files present in response to requests without the directory being shared among the nodes.

davidalger commented 6 years ago

+1 here guys! Nothing should be written to pub/static when in production mode, and a setup:static-content:deploy should write out merged files to static/_cache. This causes problems in load-balanced environments where a snapshot of a pre-compiled build is deployed.

dan-ding commented 6 years ago

bump sort of related: https://github.com/magento/magento2/issues/9542

ghost commented 6 years ago

Hi @scottsb i cannot reproduce you scenario, which version of magento you've tested ?

graemel commented 6 years ago

I see this behaviour in 2.1.

davidalger commented 6 years ago

This is a problem with 2.2.x as well. It starts with the files in static/_cache/merged not being written during static content deploy, and then the issue is compounded (on either a read-only file system or ) when Magento attempts to write merged JS / CSS files to disk during a web request.

ghost commented 6 years ago

@scottsb , we are closing this issue due to inactivity. If you'd like to update it, please reopen the issue.

ishakhsuvarov commented 6 years ago

@davidalger @scottsb Sorry for the confusion. This ticket was closed mistakenly. We will take a look at it again and let you know.

ghost commented 6 years ago

Hi @scottsb , thank you for your report. We've acknowledged the issue and added to our backlog.

kirkmadera commented 6 years ago

We also build separate from deploy and ran into this issue. We are expecting pub/static to actually be static and read only. We have also made the workaround to make the cache dir writable. Would be cleaner if these merged files generated as part of static content deploy.

scrivvles commented 6 years ago

We are experiencing this same issue on our production Magento 2.2.6 server.

gwharton commented 5 years ago

Looking forward to an update on this. Running 2.2.7 on a cluster and had to disable merging of css, not because of non writeable /pub/static/_cache folder, as I have allowed it, but individual servers don't seem to be triggering the generation of the cache files. I seem to get many different min.css files generated, with different hash's, but not always the one that the browser is requesting. I'd love to see these generated in the static:content:deploy phase.

kirkmadera commented 5 years ago

I think I understand the reasoning behind this. It's because it has to create the merged file for every possible frontend page, which would be tough to generate statically, especially without a full running app present to crawl.

I honestly think documenting in Magento docs that pub/static/_cache needs to be writable and shared across all frontend servers would be enough.

gwharton commented 5 years ago

Why shared. Does magento keep track of which ones have been generated, and in effect only generates it on one of the cluster nodes and 404s it from the rest of the cluster as all cluster nodes think its already generated, not knowing it is only present on one node.

That would certainly explain what i am seeing.

scottsb commented 5 years ago

I was mulling over this last night and was thinking of how this could be solved. This is rough and may not be workable, but it might be a start:

  1. Change the generation of the files to happen on-demand rather than as part of the triggering page request. This would allow whatever server receives the request to generate the file locally, even if it's a different server than triggered it originally. This file could then be cached using the general cache config (such as in Redis or at least in the expected var cache directory if using disk). In order for this to work the URL either has to include sufficient data or reference some cache key (which would be stored during the initial page request).

  2. The first step alone would avoid the present filesystem permissions issue but at the significant cost of requiring a Magento boot to serve the merged files, even if actual processing is quick once cached. So the next step would be changing the default Varnish VCL config to include these merged static files. (Currently all non-HTML assets are excluded by default.) This only solves the problem for those using Varnish, but that's recommended config for prod in any case.

A different, more exotic option would be to keep the merging in the initial page request but to push the data into the full page cache inline. This wouldn't be hard for Magento's built-in FPC, but for Varnish it would require us to push data out to it rather than waiting on it to cache while proxying. This is possible but is a rather non-standard approach.

scottsb commented 5 years ago

@gwharton Yes, Magento merges the file as part of the page request and saves it the local disk. It's expected that the file is then served directly by the web server as a static file from then on (it never routes through PHP/Magento). But if the merged asset request goes to a different server in the cluster, then it won't be present locally if the directory isn't shared.

gwharton commented 5 years ago

@scottsb Yeah, I just deployed a shared writable pub/static/_cache folder across my cluster and re-enabled merging of css files and all seems ok.

geerlingguy commented 5 years ago

I've also had to mount pub/static/_cache as a shared folder in my Kubernetes environment since any time more than one Pod (container) is running, the file will be generated on one, but not another, and that kinda messes things up—plus I have to make sure it's writeable (which it wasn't earlier).

geerlingguy commented 5 years ago

To follow up to the previous commit, I can't actually mount pub/static/_cache into my containers because when Magento rebuilds static assets it tries to delete the _cache directory, causing a fatal error, because NFS mounts cannot be deleted, you get a "device is busy" exception.

So the next best compromise is mounting all of pub/static on all my containers. But if you do this, make sure the .htaccess file that ships with Magento core (which has rewrite rules for the 'version' directory for assets) is copied into the shared pub/static directory, otherwise all your static assets will be available, but 404'ing because Magento loads them at paths like pub/static/version1548342577/frontend/ (note that versionXYZ is not an actual directory).

dooblem commented 5 years ago

I'm having the same problem. I will put static/_cache in shared storage, writable by www-data. static appears to be not static...

gwharton commented 5 years ago

Depending on how many modules/extensions you have, you may find it easier to disable merging of css (and thus there is no static/_cache folder) and manually streamline your css instead.

There may be limited benefit to merging css with http2 anyway.

Janagiramramakrishnan commented 5 years ago

I have enabled the merge JS & CSS, minification JS & CSS and bundled JS options. I have set up multiple instances and turned on the cloud front caching in AWS. But whenever the deployment is initiated through AWS code pipeline and merged JS and CSS files are missing and the website page is broken.

Current Result:

Expected / Actual Result:

csdougliss commented 5 years ago

What is the current status of this? I share pub/media across servers, but not pub/static.

After a deploy, I have to clear the cache on all servers otherwise CSS is not loaded.

I could add back static content deploy again in capistrano if required.

SeeniBorn commented 5 years ago

I'm also facing the same issue. We have multiple instances with AWS. Post deploy my website broken due to 404's on css and js files. Will the issue be solved when we share the pub/static/_cache across all instances ?

gwharton commented 5 years ago

@SeeniBorn

Either turn off CSS merging, or pub/static/_cache needs to be writable by and shared across all instances.

SeeniBorn commented 5 years ago

@gwharton Turning off CSS merging will affect the site performance right? If so, I will go for shared across all instances.

csdougliss commented 5 years ago

You can't share pub/static/_cache without sharing pub/static.

Is that what's needed?

gwharton commented 5 years ago

Depends on how many modules you have. I just put up with the fact that i have about 10 css files instead of 1 merged and dont need shared static. Has little effect with http/2.0 anyway. What has far more of an effect is not being able to merge js effectively as there are far far more of them.

gwharton commented 5 years ago

My pub/static folder is uploaded to a read only cdn drive at deploy time so i cant have a shared writeable area under pub/static using that deploy method. My frontends dont even have the pub/static folder on them.

SeeniBorn commented 5 years ago

What is the cdn provider you are using?

gwharton commented 5 years ago

My servers are on google compute engine so i just upload static content to a google storage bucket configured for direct public access. Handles all the caching and scaling and offloads the majority from your server. Works quite well (given a few tweaks)

csdougliss commented 5 years ago

Is this the kind of error everyone is seeing:

[2019-06-06 12:40:28] report.ERROR: Requested path '_cache/merged/3be83f6cdb1c6b4e61ac4c5ee753c201.min.css' is wrong. [] []
[2019-06-06 12:40:28] report.CRITICAL: Requested path '_cache/merged/3be83f6cdb1c6b4e61ac4c5ee753c201.min.css' is wrong. [] []
[2019-06-06 12:40:28] report.ERROR: Requested path '_cache/merged/5bebc9bda012302869ab3ecd7b2a9006.min.js' is wrong. [] []
[2019-06-06 12:40:28] report.CRITICAL: Requested path '_cache/merged/5bebc9bda012302869ab3ecd7b2a9006.min.js' is wrong. [] []
[2019-06-06 12:41:58] report.ERROR: Requested path '_cache/merged/3be83f6cdb1c6b4e61ac4c5ee753c201.min.css' is wrong. [] []
[2019-06-06 12:41:58] report.CRITICAL: Requested path '_cache/merged/3be83f6cdb1c6b4e61ac4c5ee753c201.min.css' is wrong. [] []
[2019-06-06 12:41:58] report.ERROR: Requested path '_cache/merged/5bebc9bda012302869ab3ecd7b2a9006.min.js' is wrong. [] []
[2019-06-06 12:41:58] report.CRITICAL: Requested path '_cache/merged/5bebc9bda012302869ab3ecd7b2a9006.min.js' is wrong. [] []
Thomas-Papiernik-Insign commented 5 years ago

+1. Any workaround ?

vasusundar commented 5 years ago

@SeeniBorn / @craigcarnell, Solution for this one is that we need to share the same pub/static for the different instance in AWS. Is this solution will work?.

LSERRE commented 4 years ago

+1 , This is still an issue with Magento 2.3.1

gjportegies commented 4 years ago

I'm having this issue as wel on Magento 2.3.4 after enabling CSS merge. Because of this it looks like my images are dissapearing on PDP.

erfanimani commented 4 years ago

Same here — issue is happening on 2.3.4.

hakro commented 4 years ago

Having the issue on 2.3.4 as well. The static-content deployment happens on the CI, which will build a docker image. This image does not have the pub/static/_cache directory. If I deploy a first container, it works well, and the _cache directory is generated somehow. If I deploy a second container, from exactly the same image, the _cache directory is not there, so CSS and JS fails. I think sharing the _cache is a workaround, but is not a good solution. The whole purpose of it is to increase performance, if this cache lives on NFS, it would defeat the purpose, independently of whether you have a CDN on top or not. Also, even with HTTP2, downloading a single file is always better that downloading 15 files, so disabling bundling is not ideal either. I think every instance should have it's own _cache directory, just like it's own static assets.

rioux602 commented 4 years ago

Having the issue on 2.3.4 as well. The static-content deployment happens on the CI, which will build a docker image. This image does not have the pub/static/_cache directory. If I deploy a first container, it works well, and the _cache directory is generated somehow. If I deploy a second container, from exactly the same image, the _cache directory is not there, so CSS and JS fails. I think sharing the _cache is a workaround, but is not a good solution. The whole purpose of it is to increase performance, if this cache lives on NFS, it would defeat the purpose, independently of whether you have a CDN on top or not. Also, even with HTTP2, downloading a single file is always better that downloading 15 files, so disabling bundling is not ideal either. I think every instance should have it's own _cache directory, just like it's own static assets.

+1

dev-ajaysaini commented 4 years ago

static-content:deploy command doesn't generates any file under pub/static/_cache/merged in Magento 2.4.0

gwharton commented 4 years ago

@webkul-ajaysaini You are correct. The frontend does it as and when requests come in.

trinhpham commented 3 years ago

@webkul-ajaysaini You are correct. The frontend does it as and when requests come in.

That should be a problem, which forces us to share the static folder between web nodes => the bottle-neck is becoming to the storage sharing system (we - like others - are using NFS). IMO, web nodes should have their own files system for its best performance.

nemke82 commented 3 years ago

And if a folder does not exist (pub/static/_cache), Magento creates _cache folder in var/tmp directory which grows unlimited.

var/tmp $ ls _cache

Is fix only to create the folder manually or?

harrigo commented 3 years ago

Not sure if my issue is slightly different, im on 2.3.5-p1 Community and have locked my permissions down for those folders as per DevDocs and for just one module I have the following error sometimes generated in exception.log:

[2021-04-16 08:08:58] main.CRITICAL: The path "/home/sites/magento/public_html/var/view_preprocessed/pub/static/frontend/Buzz/buzzluma/en_GB/Harrigo_CallForPrice/css/style.min.css" is not writable. {"exception":"[object] (Magento\\Framework\\Exception\\FileSystemException(code: 0): The path \"/home/sites/magento/public_html/var/view_preprocessed/pub/static/frontend/Buzz/buzzluma/en_GB/Harrigo_CallForPrice/css/style.min.css\" is not writable. at /home/sites/magento/public_html/vendor/magento/framework/Filesystem/Directory/Write.php:57)"} []

Seems the file is present in pub static and should also be merged as referenced in default_head_blocks.xml file not sure why Magento trying to write it in there?

UPDATE

Yeah i also get the same error however for pub/static/_cache which i guess is same as above.

pub/static/_cache" cannot be created Warning!mkdir(): Permission denied {

Don't really wanna open up permissions on those directories but feel I might have to unless there is a workaround?

github-jira-sync-bot commented 3 years ago

:white_check_mark: Jira issue https://jira.corp.magento.com/browse/AC-1005 is successfully created for this GitHub issue.

m2-assistant[bot] commented 3 years ago

:white_check_mark: Confirmed by @engcom-Hotel. Thank you for verifying the issue.
Issue Available: @engcom-Hotel, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

onlinebizsoft commented 3 years ago

Sometime we get empty minify JS cached on cloudfront (path is located at /pub/static/_cache) so ideally deploy-static process should build these ready

lbajsarowicz commented 2 years ago

2.4.3, 2022 issue is still there.

bilalusean commented 2 years ago

Any solution? For me, it is happening in Magento 2.4.2. I don't see any merged CSS added inside the HTML head tag. Also, it creates files in the var/tmp folder on every request. It grows to 300GB within a day and it down the server (No Space left). Right now, we schedule cron to clear the var/tmp folder to fix the "No space left" issue.

Load the home page first time

/var/tmp/_cache$ find . -type f ./merged/496119470a3f8db183209c81713b1f0e.min.css_OarcqoTd0W5G4yREWzaRbLgkzP9XwoVf **Load the same page scecond time** /var/tmp/_cache$ find . -type f ./merged/496119470a3f8db183209c81713b1f0e.min.css_OarcqoTd0W5G4yREWzaRbLgkzP9XwoVf ./merged/84ca4336623d8f0129099e328566962c.min.css_WvL2yQC0FIhpRDk8CGlSguKDe69VGWny
Stoyvo commented 2 years ago

Magento 2.4.4 - Issue still present.

CSS/JS should never be generated on web-request, these are static files, they do not change, having this part of the web request is a critical design flaw. It severely affects containerized applications and forces infrastructure work-arounds.

Merged JS / CSS should be built in static content deploy and not generated by frontend request.

mattgrul commented 2 years ago

With the current "PR Created" label being added I'm hoping we are close to a possible solution for this issue. However I can't seem to find any corresponding PR, does anyone have a link?