search

FinalsClub / karmaworld

KarmaNotes.org v3.0
GNU Affero General Public License v3.0
7 stars 6 forks source link

django compressor: is it doing anything? besides 500 crashing Django? #397

Closed btbonval closed 9 years ago

btbonval commented 9 years ago

While running a local VM, I got an error about some compress key missing and that I needed to run manage.py compress. This is a new issue, but it looks like compress has been around for some time.

The error might have been because I was running a database dumped from the beta system, but that assumes compress uses the database to store stateful information.

It sounds like compress is used to glob together js and css files in a tight format. That's great! But I've never run it manually before.

btbonval commented 9 years ago 
You have offline compression enabled, but key BLAH is missing from the offline manifest. You may need to run "python manage.py compress".

This error raises its head again after changing an environment variable in foreman.

yourcelf commented 9 years ago

django-compressor can run in two modes -- "offline" mode where you compress via a management command, and "online" mode where it compares a hash of the {% compress %}{% endcompress %} block contents to a key in the cache and recompiles the assets if needed. I presume we're using online mode. In that case, it'll use the cache to store an association between each compress block in the source and a file. Compiled assets are stored in static/CACHE, and are stored using the defualt file storage (which we have set to be S3). Checking the source on prod, it's clear that css and js are both being compressed there.

Additionally, by default, compressor doesn't minify and concatenate assets if DEBUG == True. If you want to test it locally, you can either set DEBUG = False or override the default with COMPRESS_ENABLED = True in settings.

Again based on casual perusal of the production source, it doesn't look like there are obviously compressible assets that we're missing at the moment.

yourcelf commented 9 years ago

Following up after reading more carefully: we might be using it in "offline mode" in prod, in which case the management command needs to be run. I'm not sure which key it's missing -- could be something for managing its connection to AWS or to another cache server (redis?) somewhere.

btbonval commented 9 years ago

@yourcelf I'm surprised you didn't bump into this issue, because you're also using a beta system database dump and the same branch of git code. Very confusing.

The prod.py settings enable both online and offline caching. When I disabled offline compression, I got errors about memcache. At first I thought it was just a fluke.

Now I bet that's the problem. I probably need to fix the memcache problem so that online mode works and the system doesn't need to fall back to offline mode. I bet that's the difference in our configurations.

yourcelf commented 9 years ago

I have memcache running on my system, and you probably didn't on your VM? Could explain my lack of issues.

btbonval commented 9 years ago

Yeah I thought I installed it, but maybe I only installed memcached-dev as a prereq and forgot memcached itself. We used to have automation to take care of this in the repo, but it got deleted (and thus was left unmaintained) after the switch to Heroku.

btbonval commented 9 years ago

I only installed memcached-dev but forgot memcached. It's now installed and running, but the error persists.

20:46:52 web.1  | Internal Server Error: /note/harvard/cs-50-114/1st-lecture
20:46:52 web.1  | Traceback (most recent call last):
20:46:52 web.1  |   File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 138, in get_response
...
20:46:52 web.1  |   File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/django/template/base.py", line 844, in render_node
20:46:52 web.1  |     return node.render(context)
20:46:52 web.1  |   File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 147, in render
20:46:52 web.1  |     return self.render_compressed(context, self.kind, self.mode, forced=forced)
20:46:52 web.1  |   File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 88, in render_compressed
20:46:52 web.1  |     cached_offline = self.render_offline(context, forced=forced)
20:46:52 web.1  |   File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/compressor/templatetags/compress.py", line 72, in render_offline
20:46:52 web.1  |     'You may need to run "python manage.py compress".' % key)
20:46:52 web.1  | OfflineGenerationError: You have offline compression enabled but key "95213aa91f496f23f83a46b52e763231" is missing from offline manifest. You may need to run "python manage.py compress".
20:46:52 web.1  | ERROR:django.request:Internal Server Error: /note/harvard/cs-50-114/1st-lecture

I have run python manage.py compress at least 12 times now. I'd love to know why that key in particular is being sought. I can't find that key mentioned on the file system and I don't see any obvious place for it to be stored in the database.

btbonval commented 9 years ago

I disabled offline compression in prod.py. Now the page renders. That's really strange. My assumption that online was checked first, failed, and then defaulted to offline appears incorrect.

In any case, this does not effect the production systems and I appear to have found a workaround for my use for the time being. Good enough for closing this ticket.

btbonval commented 9 years ago

ehh turns out django-compress could potentially be very useful for dealing with CDN problems in #396.

At present, I do not see anything happening with compress. All the js and css appear to be posted in the S3 bucket as-is, even though COMPRESS_ENABLED = True is set in prod.py and run on my VM, beta, and prod.

So ... the question is reiterated: does it do anything? And can we make it, because that would mesh well with our CDN.

btbonval commented 9 years ago

This configuration appears good: https://github.com/FinalsClub/karmaworld/blob/note-editing/karmaworld/settings/prod.py#L147-L151

The default storage env var is confirmed to be S3 boto on staging and my VM. The cloudfront domain on both is also correct from the environment.

btbonval commented 9 years ago

found it!

compress is pushing into the bucket under /static/CACHE/[js|css]/. I see fresh files there.

Django has been instructed to prepend /static to all file locations into and out of the static location. https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/settings/prod.py#L111-L114

It looks like these files are working correctly with compress: https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/base.html#L11-L28

As taken from http://karmanotes-beta.herokuapp.com/

  <link rel="shortcut icon" href="//d2ry5v6uki5dz0.cloudfront.net/static/img/favicon.ico">
  <link rel="stylesheet" href="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/9de46941e995.css" type="text/css" media="all" />
  <script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js"></script>

  <!-- block pagestyle -->

  <link rel="stylesheet" href="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css" type="text/css" media="all" />

  <!-- end block pagestyle -->
btbonval commented 9 years ago

So that answers the question of "Does it integrate with collectstatic so that compressed static files are actually uploaded?". Yeah, actually it does.

It appears that "Does it integrate with collectstatic so that uncompressed static files are not actually uploaded?" is not possible. All static files are uploaded whether or not they are handled by compress. So note-detail.js is uploaded even though the only call to it is inside compress.

Here's where note-detail.js is wrapped by compress (the only appearance of note-detail.js in the codebase): https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/notes/note_base.html#L31-L37

I disabled beta's cloudfront for testing. I see attempts to download the hash file names, and no attempts to download e.g. note-detail.js. This is the expected behavior, but I was not seeing it earlier.

Waiting for beta's cloudfront to come back online to test again.

btbonval commented 9 years ago

okay, cloudfront is running again, and once again I am seeing note-detail.js in the network tab of the console and in the source code view.

Is FireFox translating the hash codes into files even in the debug areas?

btbonval commented 9 years ago

answer: no. At the top of the source I'm looking at is still calling CACHE hash filenames from base.html.

btbonval commented 9 years ago

Blah this is inconsistent and annoying. The topmost base.html template is working with compress just fine, while the note_base.html template seems to be skipping compress entirely.

btbonval commented 9 years ago

Also jQuery seems to be missing.

btbonval commented 9 years ago

jQuery should be compressed after the footer. https://github.com/FinalsClub/karmaworld/blob/3fb9188be79556bd351e3659a341c9aeeb6b4926/karmaworld/templates/base.html#L45-L71

In the page source I'm looking at:

  <!-- end include footer-->

  <script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js"></script>

  <script type="text/javascript" src="//use.typekit.net/pnc8diw.js"></script>
  <script type="text/javascript">try{Typekit.load();}catch(e){}</script>
  <script>
    $(function() {
      $(document).foundation();
    });
  </script>
  <!-- block pagescripts -->

It looks like jQuery is crammed in there, but either it is not loaded at the right time or it is not in the correct namespace. I assume this is a problem with compress. https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js

btbonval commented 9 years ago

These problems do not appear on http://karmanotes-beta.herokuapp.com/note/harvard/cs-50-114/1st-lecture. It appears that jquery is crammed in the same place and even using the same cloudfront hash; that did not get updated. So it looks like missing jquery is a separate issue entirely from compress.

 <!-- end include footer-->

  <script type="text/javascript" src="https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js"></script>

  <script type="text/javascript" src="//use.typekit.net/pnc8diw.js"></script>
  <script type="text/javascript">try{Typekit.load();}catch(e){}</script>
  <script>
    $(function() {
      $(document).foundation();
    });
  </script>
  <!-- block pagescripts -->
btbonval commented 9 years ago

forked jquery issue to #400 as it now appears unrelated to compress.

btbonval commented 9 years ago

ah, some of the weirdness with compress might be due to manifest.json. While everything else is versioned, manifest.json is statically hosted but is not versioned. http://stackoverflow.com/questions/15044399/django-compressor-cloudfront-and-manifest-json

It seems like the front end never uses manifest.json, and the backend can get around caching by accessing the S3 bucket directly using the default storage.

btbonval commented 9 years ago

darnit. S3 and Cloudfront appear to match for manifest.json, so my theory seems to be wrong.

http://karma-beta.s3.amazonaws.com/static/CACHE/manifest.json

{
  "442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
  "8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
  "97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
  "a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
  "bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
  "82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
  "efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
  "86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
  "d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
  "0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
  "88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
  "d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}

https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/manifest.json

{
  "442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
  "8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
  "97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
  "a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
  "bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
  "82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
  "efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
  "86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
  "d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
  "0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
  "88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
  "d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}
btbonval commented 9 years ago

I just noticed the key changed. Earlier I pasted this:

20:46:52 web.1  | OfflineGenerationError: You have offline compression enabled but key "95213aa91f496f23f83a46b52e763231" is missing from offline manifest. You may need to run "python manage.py compress".

now I'm getting this:

22:57:31 web.1  | OfflineGenerationError: You have offline compression enabled but key "9eaf52d2274436d3f29aa9ad5398c55e" is missing from offline manifest. You may need to run "python manage.py compress".

It looks like I'll need to run foreman run python manage.py compress every time I change branches or change the code? If so, this needs to be part of deploy code if it isn't already.

btbonval commented 9 years ago

nope, didn't help. Alright, so the key has changed, but running foreman run python manage.py compress doesn't fix the problem. Perhaps that plus collectstatic are required in tandem?

btbonval commented 9 years ago

still nope.

Maybe I'll try flushing memcache again. recommended above and also on SO. http://stackoverflow.com/questions/12849571/how-come-django-compressor-still-output-a-file-when-memcached-is-used

$ foreman run python manage.py shell
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.core.cache import cache
>>> cache._cache.flush_all()
True

still no.

This offline manifest error is as mysterious as it is awful.

Alright, so this missing key problem from offline compression can be dodged by changing a setting to disable offline compression, but I'm not sure I should change the setting. It'd no longer be in sync with staging/prod.

I need to understand what django-compress is actually doing. Time to read django-compress source code to see what its trying to do and why. Maybe run a pdb trace through the compress code as it runs.

btbonval commented 9 years ago

Found the code for django-compress. Interestingly, it has been abandoned. https://code.google.com/p/django-compress/

For current users of django-compress, django-pipeline is recommended as a maintained fork of django-compress. https://github.com/cyberdelia/django-pipeline

It recommends look at a list of asset managers for other suggestions. https://www.djangopackages.com/grids/g/asset-managers/

btbonval commented 9 years ago

I am now of the impression that django-compress is causing enough problems to be replaced rather than trying to figure out this problem and fix it.

Rather than reviewing django-compress code, it might be time to take old yeller out back.

yourcelf commented 9 years ago

It's django-compressor: https://github.com/django-compressor/django-compressor

Still well maintaned.

btbonval commented 9 years ago

Oh! Good to know!

btbonval commented 9 years ago

Apparently I got that wrong in my ticket title. https://github.com/FinalsClub/karmaworld/blob/master/requirements.txt#L3

Need to fix that.

btbonval commented 9 years ago

High level discussion of offline compression, but no details about the manifest or its location. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/docs/behind-the-scenes.txt

The only place in code where the missing key error appears: https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/templatetags/compress.py#L71-L73

btbonval commented 9 years ago

It is using compressor's default_storage to store/read the manifest. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/cache.py#L68-L77

default_storage is running a Django storage manager based on the settings. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/storage.py#L92-L96

Storage is defined by the DEFAULT_FILE_STORAGE env var. https://github.com/FinalsClub/karmaworld/blob/321c9fd8be8eb3776ee56f628d2918d689ac4e2c/karmaworld/settings/prod.py#L155

This is configured for storages.backends.s3boto.S3BotoStorage on Beta, Prod, and my VM.

So the filename on the S3 static is defined by the COMPRESS_OFFLINE_MANIFEST variable. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/cache.py#L60-L62

This variable is undefined on Beta and the VM. settings.ENV_VAR is coming from django.conf.

So what's the default value?

Probably manifest.json based on what I saw on the S3.

So now the question is where is the missing key coming from, since it is missing from that manifest?

yourcelf commented 9 years ago

Default name is manfiest.json. Docs:

Since the keys are hexdigests of hashes of the file contents, if the code hasn't changed, the keys should be the same. So if you run foreman python manage.py compress --force locally, the manifest generated in karmaworld/karmaworld/static/CACHE/manifest.json should be equivalent JSON to the one generated in staging or prod.

This might help identify which of (staging, cloudfront, s3) is at issue.

btbonval commented 9 years ago

Actually I have confirmed that Cloudfront and S3 are in sync, so that will help reduce factors.

I don't have a local version of CACHE. It's strictly handled via S3boto.

yourcelf commented 9 years ago

If you do foreman python manage.py compress --force under dev environment you'll get a local one. Its contents are as follows for the tip of note-editing:

{
  "084ccea86ea76470d767c5c3c9fc1730": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/2dd7387103f5.css\" type=\"text/css\" media=\"all\" />",
  "5f1819404280800c968496dac52f7190": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/5c658ccddc8f.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/4763708f8fda.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/fc3071bc89f4.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/4166624d7846.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/974831a18dff.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/826ade1e5b98.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/d1efd45c1c68.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/0890ccd6fdaa.css\" type=\"text/css\" media=\"all\" />",
  "0d42a4e17f5186d2818ca64c2c9697ba": "<script type=\"text/javascript\" src=\"/static/CACHE/js/218abd603e50.js\"></script>",
  "d68bd04e19ad3d77d1b0e7cc16b116e6": "<script type=\"text/javascript\" src=\"/static/CACHE/js/50c293a3b65b.js\"></script>",
  "e58c9b4ad6912b1a7957ee8c01a5e55b": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/92ebe1efea0e.css\" type=\"text/css\" media=\"all\" /><link rel=\"stylesheet\" href=\"/static/CACHE/css/fee81d60a4ea.css\" type=\"text/css\" media=\"all\" />",
  "fab5de9e4b80ea2642351b5cb149ce8b": "<script type=\"text/javascript\" src=\"/static/CACHE/js/670326a5aaeb.js\"></script>",
  "c0452ba64355b1d7e5bb44a08c6a1c40": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/9bb52d917178.css\" type=\"text/css\" media=\"all\" />",
  "f4448a2c928f69dcdb9e50f8adfd326a": "<script type=\"text/javascript\" src=\"/static/CACHE/js/075ddda0920e.js\"></script>",
  "0d746acfa25219e872da7362d66e055e": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/92ebe1efea0e.css\" type=\"text/css\" media=\"all\" />",
  "474599d878e001fcfddf710341a6cec4": "<script type=\"text/javascript\" src=\"/static/CACHE/js/bb568d120290.js\"></script>",
  "f135917cb0cf5770a30b44f854fd7a24": "<script type=\"text/javascript\" src=\"/static/CACHE/js/3e47e18cb4e0.js\"></script>",
  "367bbeca1bf7546b34dd92133b688202": "<link rel=\"stylesheet\" href=\"/static/CACHE/css/013dd66694cb.css\" type=\"text/css\" media=\"all\" />"
}
yourcelf commented 9 years ago

Interesting. Visiting a note detail page on localhost with COMPRESS_ENABLED = True:

TemplateSyntaxError at /note/harvard/reason-and-faith-in-the-west/kevinkelly_whattechnologywants_ch1pdf-2-14-959998

Could not parse some characters: var| a,f=this;this.composer.sandbox.getDocument()}i.observe(this.element,"keydown",function(a){if(!a.altKey&&(a.ctrlKey|||a.metaKey)){var d=a.keyCode,e=d===b&&!a.shiftKey,g=d===b&&a.shiftKey||d===c;e?(f.undo(),a.preventDefault()):g&&(f.redo(),a.preventDefault())

Request Method:     GET
Request URL:    http://localhost:8000/note/harvard/reason-and-faith-in-the-west/kevinkelly_whattechnologywants_ch1pdf-2-14-959998
Django Version:     1.5.12
Exception Type:     TemplateSyntaxError
Exception Value:    

Could not parse some characters: var| a,f=this;this.composer.sandbox.getDocument()}i.observe(this.element,"keydown",function(a){if(!a.altKey&&(a.ctrlKey|||a.metaKey)){var d=a.keyCode,e=d===b&&!a.shiftKey,g=d===b&&a.shiftKey||d===c;e?(f.undo(),a.preventDefault()):g&&(f.redo(),a.preventDefault())

Exception Location:     /home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages/django/template/base.py in __init__, line 543
Python Executable:  /home/tc1/git/karmaworld/venv/bin/python
Python Version:     2.7.6
Python Path:    

['/home/tc1/git/karmaworld',
 '/home/tc1/git/karmaworld/venv/lib/python2.7',
 '/home/tc1/git/karmaworld/venv/lib/python2.7/plat-x86_64-linux-gnu',
 '/home/tc1/git/karmaworld/venv/lib/python2.7/lib-tk',
 '/home/tc1/git/karmaworld/venv/lib/python2.7/lib-old',
 '/home/tc1/git/karmaworld/venv/lib/python2.7/lib-dynload',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-x86_64-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages',
 '/home/tc1/git/karmaworld/venv/local/lib/python2.7/site-packages/newrelic-2.40.0.34',
 '/home/tc1/git/karmaworld/karmaworld/apps',
 '/home/tc1/git/karmaworld/karmaworld']

Server time:    Tue, 17 Feb 2015 22:51:16 -0500

Looks like there might be a problem with django-compressor parsing some of the js.

btbonval commented 9 years ago

Still no local version.

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman run python manage.py compress --force
Found 'compress' tags in:
    /vagrant/karmaworld/templates/notes/note_base.html
    /vagrant/karmaworld/templates/base.html
    /vagrant/karmaworld/templates/courses/course_list.html
    /vagrant/karmaworld/templates/terms.html
    /vagrant/karmaworld/templates/about.html
    /vagrant/karmaworld/templates/courses/course_detail.html
    /vagrant/karmaworld/templates/user_profile.html
    /vagrant/karmaworld/templates/notes/search_results.html
Compressing... done
Compressed 14 block(s) from 8 template(s).
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ find ./ -name "CACHE"
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$

Output looks the same regardless of whether or not I use --force. I've run without force a million times already, but here's another:

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman run python manage.py compress
Found 'compress' tags in:
    /vagrant/karmaworld/templates/notes/note_base.html
    /vagrant/karmaworld/templates/base.html
    /vagrant/karmaworld/templates/courses/course_list.html
    /vagrant/karmaworld/templates/terms.html
    /vagrant/karmaworld/templates/about.html
    /vagrant/karmaworld/templates/courses/course_detail.html
    /vagrant/karmaworld/templates/user_profile.html
    /vagrant/karmaworld/templates/notes/search_results.html
Compressing... done
Compressed 14 block(s) from 8 template(s).
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ find ./ -name "CACHE"
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$

Just to confirm I'm on the same branch:

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git branch
  master
  note-edit-download-s3
* note-editing
  verisign-confirm

Here are the contents on the S3 bucket:

{
  "442c18c2a05bb4d3fd13ee627a5c7417": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/febe1bc0c6e9.css\" type=\"text/css\" media=\"all\" />",
  "8814067983406d53beedaae5667ace44": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/9baa4331d306.js\"></script>",
  "97b36458b6db8b3e32dc5a8723c35913": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/cc9da0afb61c.css\" type=\"text/css\" media=\"all\" />",
  "a7faab5496fe386820911dbc1b2d1d16": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a52a868564de.js\"></script>",
  "bcb3f4d38cd2f812acfa34b2b280c967": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/b96ad19ec98b.js\"></script>",
  "82d94a4224b72da20caf3a1be3a6d54d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/ac272f84448a.css\" type=\"text/css\" media=\"all\" />",
  "efdec72c75dd83fb57fc636dc8cd4040": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/1c9098619b8b.js\"></script>",
  "86d6e666af9057070c90cfd1c6827b4b": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/c65cf523ca3b.css\" type=\"text/css\" media=\"all\" />",
  "d4b49aaae233b114a3a190733be902b5": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/a0cebb0c3c74.js\"></script>",
  "0b7b58e7f7597cee10588304e025d150": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/7c08411ba945.css\" type=\"text/css\" media=\"all\" />",
  "88e86e7fea3c44b253cf997fad40758d": "<link rel=\"stylesheet\" href=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/css/db6299073c6f.css\" type=\"text/css\" media=\"all\" />",
  "d71b198ed04ac08b1724ed508ff86911": "<script type=\"text/javascript\" src=\"https://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/js/fc79bd7d88e1.js\"></script>"
}
yourcelf commented 9 years ago

Is vagrant running in prod or dev environment for you? I'm trying in dev..

btbonval commented 9 years ago

My environment is as close to a perfect copy of production/staging as possible, so it is using prod.

Wow you're getting an entirely different error from me.

Here's the page I've been visiting which 100% of the time tosses 500 when I have offline cache enabled: /note/harvard/cs-50-114/1st-lecture

btbonval commented 9 years ago

... although I have database contents for 1st-lecture and you probably won't. Might change things :(

btbonval commented 9 years ago

PS after all the compress running and such, I just tested that link from my local server. Still 500 with the error: 03:58:25 web.1 | OfflineGenerationError: You have offline compression enabled but key "9eaf52d2274436d3f29aa9ad5398c55e" is missing from offline manifest. You may need to run "python manage.py compress".

Looking at the manifest above, it certainly is missing. I'm not sure how many times I need to run manage.py compress to get that key in there :(

btbonval commented 9 years ago

Just to confirm I'm cleanly in the branch:

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git status
On branch note-editing
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    .vagrant/
    Vagrantfile

nothing added to commit but untracked files present (use "git add" to track)
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ git log | head
commit 4736b4b0994b07623ed5562090ddec9c8b3e2f4e
Author: Charlie DeTar <cfd@media.mit.edu>
Date:   Sat Feb 14 16:27:17 2015 -0700

    Updates to sanitizer; all-in with inline HTML

     - Remove iframe and "static hosting" as a strategy for showing a note.
       Only show inline HTML.
     - Add a "format preserving" sanitizer that does XSS filtering and prep
       for inline HTML, but doesn't remove visual markup.
yourcelf commented 9 years ago

the key the error references is not in the manifest.json I generated locally.

FWIW, I run django-compressor in 'online' mode for all of my production sites. The only time the performance is any different is when generating files for the first time with an empty cache.

Did you figure out the train you were following earlier of checking whether the contents of manifest.json can change if the filename doesn't? Was that just cloudfront, or was it s3 too?

btbonval commented 9 years ago

The manifest.json I pasted above is what is currently on S3, after running compress and so forth. It turns out that it is the same manifest.json on CloudFront, so nothing has changed.

I can confirm that I am pushing code up to S3 because it is hosting the new .js files and I see current timestamps. e.g. /static/CACHE/manifest.json is showing me a timestamp of about 9-10 minuts ago.

Here's the S3 version (10 minutes old): http://karma-beta.s3.amazonaws.com/static/CACHE/manifest.json

Here's the CloudFront version: http://d2ry5v6uki5dz0.cloudfront.net/static/CACHE/manifest.json

btbonval commented 9 years ago

I wonder if I can inject a printout to determine what is the context that is generating this missing key. https://github.com/django-compressor/django-compressor/blob/369065c6a7070b9395d4bbfd6cb61da3ed59b62f/compressor/templatetags/compress.py#L66

yourcelf commented 9 years ago

My best guess: when executing manage.py compress, django-compressor is encountering and swallowing an error when trying to parse the files to compress. This is resulting in a file not getting added to the manifest. Then when running with COMPRESS_OFFLINE = True, it's looking for a file that should be there that doesn't exist.

This would explain both the missing key errors and the fun error that I got above. My error shows up if COMPRESS_OFFLINE = False, and your error shows up if COMPRESS_ONLINE = True (I can confirm that's the case locally).

btbonval commented 9 years ago

Well I output the context, but man, that's a lot of noise and very little signal. I think I might pdb this and step through what it's doing to the context to figure out what is actually being hashed.

yourcelf commented 9 years ago

Switching the wysiwyg to reference the non-minified version of its source file eliminates my TemplateSyntaxError.

btbonval commented 9 years ago

pdb had no output :(

I was hoping to dive into this interactively to determine just what is missing.

btbonval commented 9 years ago

this is the chunk that is being choked on:

04:27:01 web.1  |     <link rel="stylesheet" type="text/css" media="all" href="//d2ry5v6uki5dz0.cloudfront.net/static/css/note_course_pages.css">
04:27:01 web.1  |     <link rel="stylesheet" type="text/css" media="all" href="//d2ry5v6uki5dz0.cloudfront.net/static/css/annotator.min.css" />
04:27:01 web.1  |     <link href="//d2ry5v6uki5dz0.cloudfront.net/static/wysihtml5/toolbar.css" type="text/css" media="all" rel="stylesheet" />

wysiwyg is definitely in there, and it's the only thing "new" to this code base in general. It sounds like you found the problem, although I wish I understood why.

yourcelf commented 9 years ago

Though it's not really fixed. Commenting out the wysihtml5 js doesn't fix the problem when running with COMPRESS_OFFLINE = True -- manage.py compress runs as expected; new manifest and files generated, but still missing the manifest.

I think the TemplateSyntaxError is just bugs in the jsmin parser... but the missing manifest entry is a different problem.