Closed btbonval closed 9 years ago
You have offline compression enabled, but key BLAH is missing from the offline manifest. You may need to run "python manage.py compress".
This error raises its head again after changing an environment variable in foreman
.
django-compressor
can run in two modes -- "offline" mode where you compress via a management command, and "online" mode where it compares a hash of the {% compress %}{% endcompress %}
block contents to a key in the cache and recompiles the assets if needed. I presume we're using online mode. In that case, it'll use the cache to store an association between each compress block in the source and a file. Compiled assets are stored in static/CACHE
, and are stored using the defualt file storage (which we have set to be S3). Checking the source on prod, it's clear that css and js are both being compressed there.
Additionally, by default, compressor
doesn't minify and concatenate assets if DEBUG == True
. If you want to test it locally, you can either set DEBUG = False
or override the default with COMPRESS_ENABLED = True
in settings.
Again based on casual perusal of the production source, it doesn't look like there are obviously compressible assets that we're missing at the moment.
Following up after reading more carefully: we might be using it in "offline mode" in prod, in which case the management command needs to be run. I'm not sure which key it's missing -- could be something for managing its connection to AWS or to another cache server (redis?) somewhere.
@yourcelf I'm surprised you didn't bump into this issue, because you're also using a beta system database dump and the same branch of git code. Very confusing.
The prod.py
settings enable both online and offline caching. When I disabled offline compression, I got errors about memcache. At first I thought it was just a fluke.
Now I bet that's the problem. I probably need to fix the memcache problem so that online mode works and the system doesn't need to fall back to offline mode. I bet that's the difference in our configurations.
I have memcache running on my system, and you probably didn't on your VM? Could explain my lack of issues.
Yeah I thought I installed it, but maybe I only installed memcached-dev
as a prereq and forgot memcached
itself. We used to have automation to take care of this in the repo, but it got deleted (and thus was left unmaintained) after the switch to Heroku.
I only installed memcached-dev
but forgot memcached
. It's now installed and running, but the error persists.
20:46:52 web.1 | Internal Server Error: /note/harvard/cs-50-114/1st-lecture
20:46:52 web.1 | Traceback (most recent call last):
20:46:52 web.1 | File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 138, in get_response
...
20:46:52 web.1 | File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/django/template/base.py", line 844, in render_node
20:46:52 web.1 | return node.render(context)
20:46:52 web.1 | File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 147, in render
20:46:52 web.1 | return self.render_compressed(context, self.kind, self.mode, forced=forced)
20:46:52 web.1 | File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 88, in render_compressed
20:46:52 web.1 | cached_offline = self.render_offline(context, forced=forced)
20:46:52 web.1 | File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 72, in render_offline
20:46:52 web.1 | 'You may need to run "python manage.py compress".' % key)
20:46:52 web.1 | OfflineGenerationError: You have offline compression enabled but key "95213aa91f496f23f83a46b52e763231" is missing from offline manifest. You may need to run "python manage.py compress".
20:46:52 web.1 | ERROR:django.request:Internal Server Error: /note/harvard/cs-50-114/1st-lecture
I have run python manage.py compress
at least 12 times now. I'd love to know why that key in particular is being sought. I can't find that key mentioned on the file system and I don't see any obvious place for it to be stored in the database.
I disabled offline compression in prod.py
. Now the page renders. That's really strange. My assumption that online was checked first, failed, and then defaulted to offline appears incorrect.
In any case, this does not effect the production systems and I appear to have found a workaround for my use for the time being. Good enough for closing this ticket.
ehh turns out django-compress could potentially be very useful for dealing with CDN problems in #396.
At present, I do not see anything happening with compress. All the js and css appear to be posted in the S3 bucket as-is, even though COMPRESS_ENABLED = True
is set in prod.py
and run on my VM, beta, and prod.
So ... the question is reiterated: does it do anything? And can we make it, because that would mesh well with our CDN.
This configuration appears good: https://github.com/FinalsClub/karmaworld/blob/note-editing/karmaworld/settings/prod.py#L147-L151
The default storage env var is confirmed to be S3 boto on staging and my VM. The cloudfront domain on both is also correct from the environment.
found it!
compress is pushing into the bucket under /static/CACHE/[js|css]/
. I see fresh files there.
Django has been instructed to prepend /static
to all file locations into and out of the static location.
https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/settings/prod.py#L111-L114
It looks like these files are working correctly with compress: https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/base.html#L11-L28
As taken from http://karmanotes-beta.herokuapp.com/
<link rel="shortcut icon" href="//d2ry5v6uki5dz0.cloudfront.net/static/img/favicon.ico">
<link rel="stylesheet" href="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/9de46941e995.css" type="text/css" media="all" />
<script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js"></script>
<!-- block pagestyle -->
<link rel="stylesheet" href="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css" type="text/css" media="all" />
<!-- end block pagestyle -->
So that answers the question of "Does it integrate with collectstatic so that compressed static files are actually uploaded?". Yeah, actually it does.
It appears that "Does it integrate with collectstatic so that uncompressed static files are not actually uploaded?" is not possible. All static files are uploaded whether or not they are handled by compress. So note-detail.js
is uploaded even though the only call to it is inside compress.
Here's where note-detail.js
is wrapped by compress (the only appearance of note-detail.js
in the codebase):
https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/notes/note_base.html#L31-L37
I disabled beta's cloudfront for testing. I see attempts to download the hash file names, and no attempts to download e.g. note-detail.js
. This is the expected behavior, but I was not seeing it earlier.
Waiting for beta's cloudfront to come back online to test again.
okay, cloudfront is running again, and once again I am seeing note-detail.js
in the network tab of the console and in the source code view.
Is FireFox translating the hash codes into files even in the debug areas?
answer: no. At the top of the source I'm looking at is still calling CACHE hash filenames from base.html
.
Blah this is inconsistent and annoying. The topmost base.html
template is working with compress
just fine, while the note_base.html
template seems to be skipping compress
entirely.
Also jQuery seems to be missing.
jQuery should be compressed after the footer. https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/base.html#L45-L71
In the page source I'm looking at:
<!-- end include footer-->
<script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js"></script>
<script type="text/javascript" src="//use.typekit.net/pnc8diw.js"></script>
<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
<script>
$(function() {
$(document).foundation();
});
</script>
<!-- block pagescripts -->
It looks like jQuery is crammed in there, but either it is not loaded at the right time or it is not in the correct namespace. I assume this is a problem with compress. https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js
These problems do not appear on http://karmanotes-beta.herokuapp.com/note/harvard/cs-50-114/1st-lecture. It appears that jquery is crammed in the same place and even using the same cloudfront hash; that did not get updated. So it looks like missing jquery is a separate issue entirely from compress.
<!-- end include footer-->
<script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js"></script>
<script type="text/javascript" src="//use.typekit.net/pnc8diw.js"></script>
<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
<script>
$(function() {
$(document).foundation();
});
</script>
<!-- block pagescripts -->
forked jquery issue to #400 as it now appears unrelated to compress.
ah, some of the weirdness with compress might be due to manifest.json
. While everything else is versioned, manifest.json
is statically hosted but is not versioned.
http://stackoverflow.com/questions/15044399/django-compressor-cloudfront-and-manifest-json
It seems like the front end never uses manifest.json, and the backend can get around caching by accessing the S3 bucket directly using the default storage.
darnit. S3 and Cloudfront appear to match for manifest.json
, so my theory seems to be wrong.
http://karma-beta.s3.amazonaws.com/static/CACHE/manifest.json
{
"442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
"8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
"97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
"a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
"bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
"82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
"efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
"86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
"d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
"0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
"88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
"d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}
https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/manifest.json
{
"442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
"8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
"97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
"a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
"bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
"82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
"efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
"86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
"d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
"0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
"88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
"d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}
I just noticed the key changed. Earlier I pasted this:
20:46:52 web.1 | OfflineGenerationError: You have offline compression enabled but key "95213aa91f496f23f83a46b52e763231" is missing from offline manifest. You may need to run "python manage.py compress".
now I'm getting this:
22:57:31 web.1 | OfflineGenerationError: You have offline compression enabled but key "9eaf52d2274436d3f29aa9ad5398c55e" is missing from offline manifest. You may need to run "python manage.py compress".
It looks like I'll need to run foreman run python manage.py compress
every time I change branches or change the code? If so, this needs to be part of deploy code if it isn't already.
nope, didn't help. Alright, so the key has changed, but running foreman run python manage.py compress
doesn't fix the problem. Perhaps that plus collectstatic
are required in tandem?
still nope.
Maybe I'll try flushing memcache again. recommended above and also on SO. http://stackoverflow.com/questions/12849571/how-come-django-compressor-still-output-a-file-when-memcached-is-used
$ foreman run python manage.py shell
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.core.cache import cache
>>> cache._cache.flush_all()
True
still no.
This offline manifest error is as mysterious as it is awful.
Alright, so this missing key problem from offline compression can be dodged by changing a setting to disable offline compression, but I'm not sure I should change the setting. It'd no longer be in sync with staging/prod.
I need to understand what django-compress is actually doing. Time to read django-compress
source code to see what its trying to do and why. Maybe run a pdb trace through the compress code as it runs.
Found the code for django-compress. Interestingly, it has been abandoned. https://code.google.com/p/django-compress/
For current users of django-compress, django-pipeline is recommended as a maintained fork of django-compress. https://github.com/cyberdelia/django-pipeline
It recommends look at a list of asset managers for other suggestions. https://www.djangopackages.com/grids/g/asset-managers/
I am now of the impression that django-compress is causing enough problems to be replaced rather than trying to figure out this problem and fix it.
Rather than reviewing django-compress code, it might be time to take old yeller out back.
It's django-compressor
: https://github.com/django-compressor/django-compressor
Still well maintaned.
Oh! Good to know!
Apparently I got that wrong in my ticket title. https://github.com/FinalsClub/karmaworld/blob/master/requirements.txt#L3
Need to fix that.
High level discussion of offline compression, but no details about the manifest or its location. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/docs/behind-the-scenes.txt
The only place in code where the missing key error appears: https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/templatetags/compress.py#L71-L73
It is using compressor's default_storage to store/read the manifest. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/cache.py#L68-L77
default_storage is running a Django storage manager based on the settings. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/storage.py#L92-L96
Storage is defined by the DEFAULT_FILE_STORAGE
env var.
https://github.com/FinalsClub/karmaworld/blob/321c9fd8be8eb3776ee56f628d2918d689ac4e2c/karmaworld/settings/prod.py#L155
This is configured for storages.backends.s3boto.S3BotoStorage
on Beta, Prod, and my VM.
So the filename on the S3 static is defined by the COMPRESS_OFFLINE_MANIFEST
variable.
https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/cache.py#L60-L62
This variable is undefined on Beta and the VM. settings.ENV_VAR
is coming from django.conf
.
So what's the default value?
Probably manifest.json
based on what I saw on the S3.
So now the question is where is the missing key coming from, since it is missing from that manifest?
Default name is manfiest.json
. Docs:
Since the keys are hexdigests of hashes of the file contents, if the code hasn't changed, the keys should be the same. So if you run foreman python manage.py compress --force
locally, the manifest generated in karmaworld/karmaworld/static/CACHE/manifest.json
should be equivalent JSON to the one generated in staging or prod.
This might help identify which of (staging, cloudfront, s3) is at issue.
Actually I have confirmed that Cloudfront and S3 are in sync, so that will help reduce factors.
I don't have a local version of CACHE. It's strictly handled via S3boto.
If you do foreman python manage.py compress --force
under dev environment you'll get a local one. Its contents are as follows for the tip of note-editing
:
{
"084ccea86ea76470d767c5c3c9fc1730": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/2dd7387103f5.css\" type=\"text/css\" media=\"all\" />",
"5f1819404280800c968496dac52f7190": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/5c658ccddc8f.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/4763708f8fda.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/fc3071bc89f4.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/4166624d7846.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/974831a18dff.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/826ade1e5b98.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/d1efd45c1c68.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/0890ccd6fdaa.css\" type=\"text/css\" media=\"all\" />",
"0d42a4e17f5186d2818ca64c2c9697ba": "<script type=\"text/javascript\" src=\"/static/CACHE/js/218abd603e50.js\"></script>",
"d68bd04e19ad3d77d1b0e7cc16b116e6": "<script type=\"text/javascript\" src=\"/static/CACHE/js/50c293a3b65b.js\"></script>",
"e58c9b4ad6912b1a7957ee8c01a5e55b": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/92ebe1efea0e.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/fee81d60a4ea.css\" type=\"text/css\" media=\"all\" />",
"fab5de9e4b80ea2642351b5cb149ce8b": "<script type=\"text/javascript\" src=\"/static/CACHE/js/670326a5aaeb.js\"></script>",
"c0452ba64355b1d7e5bb44a08c6a1c40": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/9bb52d917178.css\" type=\"text/css\" media=\"all\" />",
"f4448a2c928f69dcdb9e50f8adfd326a": "<script type=\"text/javascript\" src=\"/static/CACHE/js/075ddda0920e.js\"></script>",
"0d746acfa25219e872da7362d66e055e": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/92ebe1efea0e.css\" type=\"text/css\" media=\"all\" />",
"474599d878e001fcfddf710341a6cec4": "<script type=\"text/javascript\" src=\"/static/CACHE/js/bb568d120290.js\"></script>",
"f135917cb0cf5770a30b44f854fd7a24": "<script type=\"text/javascript\" src=\"/static/CACHE/js/3e47e18cb4e0.js\"></script>",
"367bbeca1bf7546b34dd92133b688202": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/013dd66694cb.css\" type=\"text/css\" media=\"all\" />"
}
Interesting. Visiting a note detail page on localhost with COMPRESS_ENABLED = True
:
TemplateSyntaxError at /note/harvard/reason-and-faith-in-the-west/kevinkelly_whattechnologywants_ch1pdf-2-14-959998
Could not parse some characters: var| a,f=this;this.composer.sandbox.getDocument()}i.observe(this.element,"keydown",function(a){if(!a.altKey&&(a.ctrlKey|||a.metaKey)){var d=a.keyCode,e=d===b&&!a.shiftKey,g=d===b&&a.shiftKey||d===c;e?(f.undo(),a.preventDefault()):g&&(f.redo(),a.preventDefault())
Request Method: GET
Request URL: http://localhost:8000/note/harvard/reason-and-faith-in-the-west/kevinkelly_whattechnologywants_ch1pdf-2-14-959998
Django Version: 1.5.12
Exception Type: TemplateSyntaxError
Exception Value:
Could not parse some characters: var| a,f=this;this.composer.sandbox.getDocument()}i.observe(this.element,"keydown",function(a){if(!a.altKey&&(a.ctrlKey|||a.metaKey)){var d=a.keyCode,e=d===b&&!a.shiftKey,g=d===b&&a.shiftKey||d===c;e?(f.undo(),a.preventDefault()):g&&(f.redo(),a.preventDefault())
Exception Location: /home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages/django/template/base.py in __init__, line 543
Python Executable: /home/tc1/git/karmaworld/venv/bin/python
Python Version: 2.7.6
Python Path:
['/home/tc1/git/karmaworld',
'/home/tc1/git/karmaworld/venv/lib/python2.7',
'/home/tc1/git/karmaworld/venv/lib/python2.7/plat-x86_64-linux-gnu',
'/home/tc1/git/karmaworld/venv/lib/python2.7/lib-tk',
'/home/tc1/git/karmaworld/venv/lib/python2.7/lib-old',
'/home/tc1/git/karmaworld/venv/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages',
'/home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages/newrelic-2.40.0.34',
'/home/tc1/git/karmaworld/karmaworld/apps',
'/home/tc1/git/karmaworld/karmaworld']
Server time: Tue, 17 Feb 2015 22:51:16 -0500
Looks like there might be a problem with django-compressor
parsing some of the js.
Still no local version.
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman run python manage.py compress --force
Found 'compress' tags in:
/vagrant/karmaworld/templates/notes/note_base.html
/vagrant/karmaworld/templates/base.html
/vagrant/karmaworld/templates/courses/course_list.html
/vagrant/karmaworld/templates/terms.html
/vagrant/karmaworld/templates/about.html
/vagrant/karmaworld/templates/courses/course_detail.html
/vagrant/karmaworld/templates/user_profile.html
/vagrant/karmaworld/templates/notes/search_results.html
Compressing... done
Compressed 14 block(s) from 8 template(s).
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ find ./ -name "CACHE"
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$
Output looks the same regardless of whether or not I use --force
. I've run without force a million times already, but here's another:
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman run python manage.py compress
Found 'compress' tags in:
/vagrant/karmaworld/templates/notes/note_base.html
/vagrant/karmaworld/templates/base.html
/vagrant/karmaworld/templates/courses/course_list.html
/vagrant/karmaworld/templates/terms.html
/vagrant/karmaworld/templates/about.html
/vagrant/karmaworld/templates/courses/course_detail.html
/vagrant/karmaworld/templates/user_profile.html
/vagrant/karmaworld/templates/notes/search_results.html
Compressing... done
Compressed 14 block(s) from 8 template(s).
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ find ./ -name "CACHE"
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$
Just to confirm I'm on the same branch:
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git branch
master
note-edit-download-s3
* note-editing
verisign-confirm
Here are the contents on the S3 bucket:
{
"442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
"8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
"97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
"a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
"bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
"82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
"efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
"86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
"d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
"0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
"88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
"d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}
Is vagrant running in prod or dev environment for you? I'm trying in dev..
My environment is as close to a perfect copy of production/staging as possible, so it is using prod.
Wow you're getting an entirely different error from me.
Here's the page I've been visiting which 100% of the time tosses 500 when I have offline cache enabled: /note/harvard/cs-50-114/1st-lecture
... although I have database contents for 1st-lecture and you probably won't. Might change things :(
PS after all the compress running and such, I just tested that link from my local server. Still 500 with the error: 03:58:25 web.1 | OfflineGenerationError: You have offline compression enabled but key "9eaf52d2274436d3f29aa9ad5398c55e" is missing from offline manifest. You may need to run "python manage.py compress".
Looking at the manifest above, it certainly is missing. I'm not sure how many times I need to run manage.py compress
to get that key in there :(
Just to confirm I'm cleanly in the branch:
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git status
On branch note-editing
Untracked files:
(use "git add <file>..." to include in what will be committed)
.vagrant/
Vagrantfile
nothing added to commit but untracked files present (use "git add" to track)
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git log | head
commit 4736b4b0994b07623ed5562090ddec9c8b3e2f4e
Author: Charlie DeTar <cfd@media.mit.edu>
Date: Sat Feb 14 16:27:17 2015 -0700
Updates to sanitizer; all-in with inline HTML
- Remove iframe and "static hosting" as a strategy for showing a note.
Only show inline HTML.
- Add a "format preserving" sanitizer that does XSS filtering and prep
for inline HTML, but doesn't remove visual markup.
the key the error references is not in the manifest.json I generated locally.
FWIW, I run django-compressor in 'online' mode for all of my production sites. The only time the performance is any different is when generating files for the first time with an empty cache.
Did you figure out the train you were following earlier of checking whether the contents of manifest.json
can change if the filename doesn't? Was that just cloudfront, or was it s3 too?
The manifest.json
I pasted above is what is currently on S3, after running compress and so forth. It turns out that it is the same manifest.json
on CloudFront, so nothing has changed.
I can confirm that I am pushing code up to S3 because it is hosting the new .js files and I see current timestamps. e.g. /static/CACHE/manifest.json
is showing me a timestamp of about 9-10 minuts ago.
Here's the S3 version (10 minutes old): http://karma-beta.s3.amazonaws.com/static/CACHE/manifest.json
Here's the CloudFront version: http://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/manifest.json
I wonder if I can inject a printout to determine what is the context that is generating this missing key. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/templatetags/compress.py#L66
My best guess: when executing manage.py compress
, django-compressor is encountering and swallowing an error when trying to parse the files to compress. This is resulting in a file not getting added to the manifest. Then when running with COMPRESS_OFFLINE = True
, it's looking for a file that should be there that doesn't exist.
This would explain both the missing key errors and the fun error that I got above. My error shows up if COMPRESS_OFFLINE = False
, and your error shows up if COMPRESS_ONLINE = True
(I can confirm that's the case locally).
Well I output the context, but man, that's a lot of noise and very little signal. I think I might pdb this and step through what it's doing to the context to figure out what is actually being hashed.
Switching the wysiwyg to reference the non-minified version of its source file eliminates my TemplateSyntaxError
.
pdb had no output :(
I was hoping to dive into this interactively to determine just what is missing.
this is the chunk that is being choked on:
04:27:01 web.1 | <link rel="stylesheet" type="text/css" media="all" href="//d2ry5v6uki5dz0.cloudfront.net/static/css/note_course_pages.css">
04:27:01 web.1 | <link rel="stylesheet" type="text/css" media="all" href="//d2ry5v6uki5dz0.cloudfront.net/static/css/annotator.min.css" />
04:27:01 web.1 | <link href="//d2ry5v6uki5dz0.cloudfront.net/static/wysihtml5/toolbar.css" type="text/css" media="all" rel="stylesheet" />
wysiwyg is definitely in there, and it's the only thing "new" to this code base in general. It sounds like you found the problem, although I wish I understood why.
Though it's not really fixed. Commenting out the wysihtml5 js doesn't fix the problem when running with COMPRESS_OFFLINE = True
-- manage.py compress
runs as expected; new manifest and files generated, but still missing the manifest.
I think the TemplateSyntaxError
is just bugs in the jsmin parser... but the missing manifest entry is a different problem.
While running a local VM, I got an error about some compress key missing and that I needed to run
manage.py compress
. This is a new issue, but it looks like compress has been around for some time.The error might have been because I was running a database dumped from the beta system, but that assumes
compress
uses the database to store stateful information.It sounds like compress is used to glob together js and css files in a tight format. That's great! But I've never run it manually before.
collectstatic
so that compressed static files are actually uploaded? [Yes]collectstatic
so that uncompressed static files are not actually uploaded? [No, all files are uploaded regardless of compress]