search

jazzband / django-silk

Silky smooth profiling for Django
MIT License
4.46k stars 335 forks source link

Incorrect string value: '\xCE\xBB, \xCF\x86...' for column 'raw_body' at row 1 #21

Closed makmanalp closed 10 years ago

makmanalp commented 10 years ago

Happens when I visit any page after installing silk.


Environment:

Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.6.5
Python Version: 2.7.6
Installed Applications:
('django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django.contrib.sitemaps',
 'observatory',
 'blog',
 'silk')
Installed Middleware:
('silk.middleware.SilkyMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.locale.LocaleMiddleware')

Traceback:
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  199.                 response = middleware_method(request, response)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/middleware.py" in process_response
  96.             self._process_response(response)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/middleware.py" in _process_response
  86.                 silk_response.save()
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in save
  545.                        force_update=force_update, update_fields=update_fields)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in save_base
  573.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in _save_table
  635.                                       forced_update)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/base.py" in _do_update
  679.         return filtered._update(values) > 0
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/query.py" in _update
  510.         return query.get_compiler(self.db).execute_sql(None)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  980.         cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/srv/atlas/env/local/lib/python2.7/site-packages/silk/sql.py" in execute_sql
  42.             return self._execute_sql(*args, **kwargs)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  786.         cursor.execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
  69.             return super(CursorDebugWrapper, self).execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
  53.                 return self.cursor.execute(sql, params)
File "/srv/atlas/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py" in execute
  124.             return self.cursor.execute(query, args)
File "/srv/atlas/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py" in execute
  207.         if not self._defer_warnings: self._warning_check()
File "/srv/atlas/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py" in _warning_check
  117.                     warn(w[-1], self.Warning, 3)

Exception Type: Warning at /
Exception Value: Incorrect string value: '\xCE\xBB, \xCF\x86...' for column 'raw_body' at row 1

raw_body seems to contain the page content, the generated sql looks like this:


u'UPDATE `silk_response` SET `request_id` = 4, `status_code` = 200, `raw_body` = <!doctype html>\n<html lang="en">\n<head>\n  <meta charset="utf-8">\n  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n\n  <base href="/">\n\n  <title>The Atlas of Economic Complexity | Home</title>\n  <meta name="description" content="Networks in Macro Economics with World Trade Data">\n  <link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="Atlas of Economic Complexity" />\n\n  <link rel="shortcut icon" type="image/x-icon" href="media/img/favicon.ico">\n\n  <!-- Load non-standard fonts -->\n  <link href=\'http://fonts.googleapis.com/css?family=Open+Sans+Condensed:300\' rel=\'stylesheet\' type=\'text/css\'>\n  <link href=\'http://fonts.googleapis.com/css?family=PT+Sans+Narrow\' rel=\'stylesheet\' type=\'text/css\' />\n  <link href=\'http://fonts.googleapis.com/css?family=Cardo\' rel=\'stylesheet\' type=\'text/css\' />\n  <link rel="stylesheet" href="media/css/normalize.css">\n  <link rel="stylesheet" href="media/css/style.css">\t\n  <link rel="stylesheet" href="media/js/libs/chosen/chosen.css" />\n\n  <!--[if lt IE 9]>\n    <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>\n  <![endif]-->\n\n   \n\n<style>\n\n.carousel {\n  position: relative;\n}\n\n.carousel-inner {\n  position: relative;\n  width: 100%;\n  overflow: hidden;\n}\n\n.carousel-inner > .item {\n  position: relative;\n  display: none;\n  -webkit-transition: 0.6s ease-in-out left;\n          transition: 0.6s ease-in-out left;\n}\n\n.carousel-inner > .item > img,\n.carousel-inner > .item > a > img {\n  display: block;\n  height: auto;\n  max-width: 100%;\n  line-height: 1;\n}\n\n.carousel-inner > .active,\n.carousel-inner > .next,\n.carousel-inner > .prev {\n  display: block;\n}\n\n.carousel-inner > .active {\n  left: 0;\n}\n\n.carousel-inner > .next,\n.carousel-inner > .prev {\n  position: absolute;\n  top: 0;\n  width: 100%;\n}\n\n.carousel-inner > .next {\n  left: 100%;\n}\n\n.carousel-inner > .prev {\n  left: -100%;\n}\n\n.carousel-inner > .next.left,\n.carousel-inner > .prev.right {\n  left: 0;\n}\n\n.carousel-inner > .active.left {\n  left: -100%;\n}\n\n.carousel-inner > .active.right {\n  left: 100%;\n}\n\n.carousel-control {\n  position: absolute;\n  top: 0;\n  bottom: 0;\n  left: 0;\n  width: 15%;\n  font-size: 50px;\n  color: #ffffff;\n  text-align: center;\n/*  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);\n  opacity: 0.5;\n  filter: alpha(opacity=50);\n  */\n}\n\n.carousel-control.left {\n  /*background-color:black;*/\n  opacity: 0;\n  background-repeat: repeat-x;\n  /*\n  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr=\'#80000000\', endColorstr=\'#00000000\', GradientType=1);\n  */\n}\n\n.carousel-control.right {\n  right: 0;\n  left: auto;\n  /*background-color:black;*/\n  opacity: 0;\n  background-repeat: repeat-x;\n  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr=\'#00000000\', endColorstr=\'#80000000\', GradientType=1);\n}\n\n\n.carousel-control:hover,\n.carousel-control:focus {\n  color: black;\n  text-decoration: none;\n  opacity: 0.9;\n\n  margin-top:-100px;\n  /*\n  filter: alpha(opacity=90);\n   */\n}\n\n.carousel-control .icon-prev,\n.carousel-control .icon-next,\n.carousel-control .glyphicon-chevron-left,\n.carousel-control .glyphicon-chevron-right {\n  position: absolute;\n  top: 50%;\n  left: 50%;\n  z-index: 5;\n  display: inline-block;\n}\n\n.carousel-control .icon-prev,\n.carousel-control .icon-next {\n  width: 20px;\n  height: 20px;\n  margin-top: -10px;\n  margin-left: -10px;\n  font-family: serif;\n}\n\n.carousel-control .icon-prev:before {\n  content: \'\\2039\';\n  font-size: 100px;\n  color:gray;\n}\n\n.carousel-control .icon-next:before {\n  content: \'\\203a\';\n  font-size: 100px;\n  color:gray;\n}\n\n.carousel-indicators {\n  position: absolute;\n  bottom: 10px;\n  left: 50%;\n  z-index: 15;\n  width: 60%;\n  padding-left: 0;\n  margin-left: -30%;\n  text-align: center;\n  list-style: none;\n}\n\n.carousel-indicators li {\n  display: inline-block;\n  width: 10px;\n  height: 10px;\n  margin: 1px;\n  text-indent: -999px;\n  cursor: pointer;\n  border: 1px solid #ffffff;\n  border-radius: 10px;\n}\n\n.carousel-indicators .active {\n  width: 12px;\n  height: 12px;\n  margin: 0;\n  background-color: lightgray;\n}\n\n.carousel-caption {\n  position: absolute;\n  bottom: 0px;\n  z-index: 10;\n  padding-top: 0px;\n  padding-bottom: 0px;\n  color: #ffffff;\n  text-align: center;\n /* text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);*/\n  padding-left:0px;\n  margin:0px; background-color: white; opacity:.8; width:300px; left:0px; \n}\n\n.carousel-caption .btn {\n  text-shadow: none;\n}\n\n@media screen and (min-width: 768px) {\n  .carousel-control .icon-prev,\n  .carousel-control .icon-next {\n    width: 30px;\n    height: 30px;\n    margin-top: -15px;\n    margin-left: -15px;\n    font-size: 30px;\n  }\n  .carousel-caption {\n    right: 0%;\n    left: 0%;\n    width: 100%;\n   \n  }\n  .carousel-indicators {\n    bottom: 20px;\n  }\n}\n\n.products-tmb, a.products-tmb {\n  float:left;\n  width: 50px;\n  height: 50px;\n  margin: 5px;\n  width: 30px; \n  height:30px; \n}\n\n.products-tmb:hover {\n  border: solid 2px #ccc;\n\n}\n\n#main #content .didyouknow_container{\n\theight: 415px;\n}\n.app_title {\n  display: none;\n}\n#main {\n  border: none;\n}\n\n#main #content {\n  border-radius: 10px;\n  margin: 10px 0 0 0;\n  width: 998px;\n}\n\n#main #content .didyouknow_container{\n  height: 450px;\n}\n\n#main #content .didyouknow{\n  display: none;\n}\n\n#main #content .headline {\n  background: rgba(255, 255, 255, 0.7);\n  margin: 0 0 10px 0;\n  overflow: hidden;\n  width: 100%;\n}\n#main #content .headline img.pin{\n  float: left;\n  margin: 0 25px 0 0;\n  width: 80px;\n}\n#main #content .headline h2{\n  font-size: 45px;\n  font-weight: 100;\n  float: left;\n  margin: 0;\n  width: 400px;\n  text-transform: uppercase;\n}\n\n#main #content img.pin{\n  float: left;\n  margin: 0 10px 0 0;\n  width: 80px;\n}\n#main #content h2{\n  font-size: 50px;\n  font-weight: 100;\n  float: left;\n  margin: 0;\n  width: 260px;\n  text-transform: uppercase;\n}\n\n#main #content .text{\n  float: left;\n  margin: 0 30px 0 0;\n  width: 365px;\n}\n#main #content .text h3{\n  line-height: 32px;\n  margin: 20px 0 10px 0;\n  font-size: 32px;\n  font-weight: 100;\n}\n#main #content .text p{\n  font-size: 22px;\n}\n#main #content .text ul{\n  font-size: 18px;\n  list-style: circle;\n  line-height: 26px;\n}\n#main #content .app {\n  margin: 10px 0 0 0;\n}\n#main #content .app h3{\n  font-size: 28px;\n  font-weight: normal;\n  margin: 0 0 0 0;\n  text-align: center;\n}\n#main #content .other_apps h3{\n  font-size: 28px;\n  font-weight: normal;\n  margin: 0px 0 10px 0;\n}\n#main #content .other_apps div{\n  float: left;\n  margin: 0 27px 15px 0;\n}\n#main #content .other_apps div a {\n  display: inline-block;\n  border: solid 2px white;\n  padding: 10px;\n}\n#main #content .other_apps div a:hover{\n  border: solid 2px #000;\n}\n#main #content .other_apps div a.credits:hover{\n  border: solid 2px white;\n}\n#main #content .other_apps div img{\n  width: 150px;\n}\n#main #content .other_apps div:last-child{\n  margin: 0;\n}\n\n/* New stuff */\n.other_apps:hover {\n   border: solid 2px gray;\n}\n\n.other_apps_first:hover {\n  background-color:white ;\n   border: solid 2px black;\n\n}\n\n\n.other_apps {\n\n  border: 1px solid white;\n}\n\n.earth_teaser {\n  background:url(\'media/img/home/teaser_earth_fade.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\n\n.earth_teaser:hover {\n  background:url(\'media/img/home/teaser_earth.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\n\n.products_teaser:hover {\n  background:url(\'media/img/home/teaser_products.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}     \n\n.products_teaser {\n  background:url(\'media/img/home/teaser_products_fade.png\') no-repeat;\n  background-size:420px 300px;\n  margin-left:9px; \n  border:2px solid white; \n  margin-top:10px;\n  width:420px;\n  height:300px;\n}      \n\nfooter.credits, footer.logos {\n  margin-left:0px;\n}\n\n</style>\n\n<!-- Hidden Modal Window for Video -->\n\n  <div id="videoModal">\n    <a href="#" id="exit">Close</a>\n    <div id="videoModalInner">\n        <iframe width="853" height="480" src="//www.youtube.com/embed/0JC24CBVsdo" frameborder="0" allowfullscreen></iframe>\n    </div>\n  </div>\n  \n</head>\n<body>\n  <div id="container">\n    <header>\n      <div class="logo" style="background-color: #fafafa; width:205px; height:66px;">\n        <a href="" title="Home" style="text-decoration: none;">\n        <h1 style="background: url(media/img/all/logo_rainbow.png) repeat-x 100% 100%; padding-bottom:22px; color: black; font-size:53px;">The Atlas</h1>\n        <h2 style="font-size:18px; margin-top:-33px; text-align:center; margin-left:0px; letter-spacing:1px; color:white; text-transform:uppercase; text-decoration:none; background-color: #a0a6a9;">of Economic Complexity</h2></a> \n      </div>  \n      <nav>\n        <ul>\n          <li><a href="explore/">Explore</a></li>\n          <li><a href="stories">Stories</a></li>\n          <li><a href="rankings">Rankings</a></li>\n          <li><a id="newBook" href="book">New Book</a></li>\n          <li><a href="about">About</a></li>\n        </ul>\n      </nav> \n      <br class="clear">\n    </header>\n\n    <div id="main" role="main" style="margin-top:70px">\n      <div class="app_title" id="title" ><h2 style="margin-left:300px; width:800px" id="text_title"></h2></div>\n\n    <style>\n      #content  h3, #content h2, h4 {\n        text-transform: uppercase;\n      }\n\n      #content h3:not(:first-child), #content h4:not(:first-child) {\n        margin: 50px 0px 0px 0px;\n      }\n    </style>\n      <!-- main content -->\n      \n  <div id="content">\n\n  <div id="carousel-example-generic" class="carousel slide" style="width:535px; height:300px;float:left; margin-left:20px; margin-top:20px; margin-right:15px; border:solid 1px #dddddd;">\n  <!-- Indicators -->\n  <!--\n  <ol class="carousel-indicators">\n    <li data-target="#carousel-example-generic" data-slide-to="0" class="active"></li>\n    <li data-target="#carousel-example-generic" data-slide-to="1"></li>\n    <li data-target="#carousel-example-generic" data-slide-to="2"></li>\n  </ol>\n  ->>\n  <!-- Wrapper for slides -->\n  <div class="carousel-inner">\n    <div class="item active">\n      <a href="explore/product_space/export/usa/all/show/2012/">\n        <img src="media/img/home/us_imports_ps.png" alt="..." style="width:550px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">What did the United States import in 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    <div class="item">\n      <a href="explore/stacked/export/usa/all/show/1996.2011.1/">\n        <img src="media/img/home/us_exports.png" alt="..." style="width:540px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">What did the United States export between 1996 and 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    <!--\n    <div class="item">\n      <a href="explore/map/export/show/all/0101/2011/">\n        <img src="media/img/home/horses_export.png" alt="..." style="width:550px; height:300px">\n        <div class="carousel-caption">\n          <h3 style="font-size:22px; color:gray;">Who exported Horses in 2011?</h3>\n          <p></p>\n        </div>\n      </a>\n    </div>\n    -->\n  </div>\n\n  <!-- Controls -->\n  <a class="left carousel-control" href="#carousel-example-generic" data-slide="prev">\n    <span class="icon-prev"></span>\n  </a>\n  <a class="right carousel-control" href="#carousel-example-generic" data-slide="next">\n    <span class="icon-next"></span>\n  </a>\n</div>\n\n    <div style="float:left; width:350px; margin-bottom:30px; font-size: 30px; font-style:italic; background: url(media/img/home/quote.jpg) no-repeat; background-size:100px; padding-left:40px; padding-top:40px; color:gray">\n      Explore Growth Opportunities\n    </div>\n\n  <a id="videoLaunch" href="#">\n\n    <div class="other_apps_first other_apps" style="float:left;width:370px; height:200px; padding:5px; margin-left:20px;border:1px solid lightgray; background-color:#fafafa;">\n     \n      <div style="width:210px; height:160px; text-align:justify;">\n        <h3 style="text-transform: none; font-size:30px">What is  <span style="text-transform: uppercase;">THE ATLAS?</span></h3> \n        <p style="text-transform: none; font-size:16px">This interactive tool can generate more than 20 million visualizations and map the path of diversification and prosperity for 128 countries. Learn more about the theory of Economic Complexity.</p>\n      </div>\n      <div style="background:url(\'media/img/home/teaser_video_obs.png\') no-repeat; background-size:130px 130px; width:130px; height:160px; margin-top:10px"></div>   \n      \n    </div>\n  \n  </a>\n\n  <p style="clear:both"></p>\n\n    <div class="other_apps" style="float:left; width:455px; margin:20px; border: solid 1px lightgray;">\n  \n  <a href="explore/tree_map/export/usa/all/show/2010/">    <h3 style="float:left; margin-left:50px;font-size:30px; text-transform: none;">Browse Countries \xbb</h3>\n\n      <!--<div class="earth_teaser" style="margin-top:0px"></div>-->\n      <div id="map" style="width:450px; height:250px"></div>\n      \n   \n  </a>\n   <div style="text-align:right; float:right; color:lightgray;" >Map transition by <a href="http://www.jasondavies.com/maps/transition/" style="text-decoration:none;color:lightgray; padding:0px; padding-top:10px" class="credits">Jason Davies</a></div>\n   </div> \n \n<style>\n\n\n\n\n</style>\n\n    <div class="other_apps" style="float:left; width:455px; margin:20px; border: solid 1px lightgray;">\n      <a href="explore/tree_map/export/show/all/8703/2010/"><h3 style="float:left; margin-left:50px;font-size:30px; text-transform: none;">Browse Products \xbb</h3></a><p style="clear:both">\n      <div style="padding-left:20px; padding-bottom:30px">\n        <a href="explore/tree_map/export/show/all/0106/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_0.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_10.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8509/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_11.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8802/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_12.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_13.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_14.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_15.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_20.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_21.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_22.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_30.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_31.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_40.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_41.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/4704/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_42.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/1121/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_43.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/5417/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_44.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_50.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_51.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/2709/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_52.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_54.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_60.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/8703/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_61.png\') no-repeat; background-size:50px;"></a>\n        <a href="explore/tree_map/export/show/all/1001/2010/" class="products-tmb" style="background: url(\'media/img/icons/community_70.png\') no-repeat; background-size:50px;"></a>\n        \n      </div>\n    </div> \n\n\n\n\n    </div> <!-- END #main -->     \n\n  </div> <!-- END #container -->\n\n  <script src="media/js/libs/jquery/jquery-1.10.2.min.js"></script>\t\n\t<script src="media/js/libs/chosen/chosen.jquery.min.js" ></script>\n  <script src="media/js/libs/bootstrap/js/bootstrap.js"></script>\n\n  <script>\n\t\n  \t// language select dropdown\n  \t$(".language_select select").chosen();\n  \t$(".language_select .chzn-search").hide();\n  \t$(".language_select select").chosen().change(function(){\n  \t\t// set session data to new language\n  \t\tvar language = $(this).val();\n  \t\twindow.location = "/set_language/"+language+"/";\n  \t});\n  \t\n    // google analytics\n  \n    (function(i,s,o,g,r,a,m){i[\'GoogleAnalyticsObject\']=r;i[r]=i[r]||function(){\n    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),\n    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)\n    })(window,document,\'script\',\'//www.google-analytics.com/analytics.js\',\'ga\');\n\n    ga(\'create\', \'UA-41291966-1\', \'harvard.edu\');\n    ga(\'send\', \'pageview\');\n\n    // current nav styling\n  \t$(function(){\n  \t\tvar current_page = document.location.pathname.split("/")[1];\n  \t\t$("nav ul a").each(function(i, el){\n  \t\t\tif($(el).text().toLowerCase() == current_page){\n  \t\t\t\t$(el).addClass("current");\n  \t\t\t}\n  \t\t})\n  \t})\n\t</script>\n\n<style>\n#sticky {\n  padding: 0.5ex;\n  width: 600px;\n  border-radius: 0.5ex;\n}\n#sticky.stick {\n  position: fixed;\n  top: 0;\n  z-index: 10000;\n  border-radius: 0 0 0.5em 0.5em;\n}\n</style>\n\n<script>\nfunction sticky_relocate() {\n  var window_top = $(window).scrollTop();\n  var div_top = $(\'#sticky-anchor\').offset().top;\n  if (window_top > div_top) {\n    $(\'#sticky\').addClass(\'stick\');\n  } else {\n    $(\'#sticky\').removeClass(\'stick\');\n  }\n}\n\n$(function() {\n  $(window).scroll(sticky_relocate);\n  sticky_relocate();\n});\n</script>\n\n\t\n<script src="media/js/libs/jquery/jquery-1.10.2.js"></script>\n<script type="text/javascript">\n    $(document).ready(function(){\n      $("#videoLaunch").click(function(){\n        $("#videoModal").css("display","block");\n        $("#videoModal").animate({opacity:1.0},400);        \n      });\n      $("#exit, #videoModal").click(function(){\n        $("#videoModal").animate({opacity:0.0},200,function(){\n          $("#videoModal").css("display", "none");\n        });\n      });\n    });\n</script>\n<script>\n  // Use to be: Rotate featured video randomly\n  // -> Will be re-used for showing stories randomly\n  // var vid = Math.round(Math.random());\n // var vid = 0;\n //  $(".didyouknow").eq(vid).show();\n  // Set explore URL to be the default country aka where the \n  // user\'s IP address says they are\n // $("nav a").eq(0).attr("href", "explore/tree_map/export/usa/all/show/2012/")\n</script>\n\n\n<script src="media/js/libs/d3/d3.v3.min.js"></script>\n<script src="media/js/libs/d3/d3.geo.projection.min.js"></script>\n<script src="media/js/libs/d3/topojson.min.js"></script>\n<script src="media/js/libs/bootstrap/js/bootstrap.js"></script>\n\n<script>\n$(function() {\n\n  $(\'#myModal\').modal({\n    show: true,\n    keyboard: false\n    });\n\n  $("#carousel-example-generic").carousel({ interval: 2000 });\n\n});\n</script>\n\n<script>\n\nvar ratio = window.devicePixelRatio || 1,\n    width = 450 * ratio,\n    height = 250 * ratio;\n\nvar options = [\n  {name: "Van der Grinten IV", projection: d3.geo.vanDerGrinten4().scale(120)}\n];\n\nvar initial = [-71, 42, 0],\n    rotate = initial.slice(),\n    pause = 0,\n    velocity = [.010, -.002],\n    t0 = Date.now();\n\nvar \n    i = 0,\n    n = options.length - 1;\n\nvar projection = d3.geo.vanDerGrinten4().scale(127); //options[i].projection;\n\nvar graticule = d3.geo.graticule()(),\n    land,\n    boundaries;\n\nvar canvas = d3.select("#map").append("canvas")\n    .attr("width", width)\n    .attr("height", height)\n    .style("width", width / ratio + "px")\n    .style("height", height / ratio + "px")\n    .call(d3.behavior.drag()\n      .origin(function() { return {x: rotate[0], y: -rotate[1]}; })\n      .on("dragstart", function() { pause |= 1; })\n      .on("drag", function(d) {\n        rotate[0] = d3.event.x;\n        rotate[1] = -d3.event.y;\n        projection.rotate(rotate);\n        redraw(path);\n      })\n      .on("dragend", function() {\n        pause &= ~1;\n        t0 = Date.now();\n        initial = rotate.slice();\n      }));\n\nvar context = canvas.node().getContext("2d");\ncontext.fillStyle = "#f9f9f9";\ncontext.strokeStyle = "#000";\n\nvar path = d3.geo.path()\n    .projection(projection)\n    .context(context);\n\nvar menu = d3.select("#projection-menu")\n    .on("change", change);\n\nmenu.selectAll("option")\n    .data(options)\n  .enter().append("option")\n    .text(function(d) { return d.name; });\n\n\nfunction change() {\n  clearInterval(interval);\n  update(options[this.selectedIndex]);\n}\n\nfunction pathTween(projection0, projection1, rotate) {\n  projection0.rotate([0, 0, 0]);\n  projection1.rotate([0, 0, 0]);\n  var t = 0,\n      projection = d3.geo.projection(function(\u03bb, \u03c6) {\n          \u03bb *= 180 / Math.PI, \u03c6 *= 180 / Math.PI;\n          var p0 = projection0([\u03bb, \u03c6]), p1 = projection1([\u03bb, \u03c6]);\n          return [(1 - t) * p0[0] + t * p1[0], (1 - t) * -p0[1] + t * -p1[1]];\n        })\n        .rotate(rotate)\n        .scale(1)\n        .translate([width / 2, height / 2])\n        .clipExtent(projection0.clipExtent()),\n      path = d3.geo.path().projection(projection).context(context);\n  return function() {\n    return function(u) {\n      t = u;\n      redraw(path);\n    };\n  };\n}\n\nfunction update(option) {\n  pause |= 2;\n  canvas.transition()\n      .duration(750)\n      .tween("path", pathTween(projection, projection = option.projection, initial = rotate.slice()))\n      .each("end", function() {\n        pause &= ~2;\n        t0 = Date.now();\n      });\n  path.projection(projection);\n}\n\nd3.timer(function() {\n  if (pause) return;\n  var t = Date.now() - t0;\n  rotate[0] = initial[0] + velocity[0] * t;\n  rotate[1] = initial[1] + velocity[1] * t;\n  projection.rotate(rotate);\n  redraw(path);\n});\n\nfunction redraw(path) {\n  context.clearRect(0, 0, width, height);\n  context.lineWidth = .5 * ratio;\n  if (land) {\n    context.strokeStyle = "#000";\n    context.beginPath(), path(land), context.fill(), context.stroke();\n    context.beginPath(), path(boundaries), context.stroke();\n  }\n  context.strokeStyle = "#999";\n  context.beginPath(), path(graticule), context.stroke();\n  context.lineWidth = 2.5 * ratio, context.strokeStyle = "#000";\n  context.beginPath(), path({type: "Sphere"}), context.stroke();\n}\n\nd3.json("media/js/libs/d3/world-110m.json", function(error, world) {\n  land = topojson.feature(world, world.objects.land);\n  boundaries = topojson.mesh(world, world.objects.countries, function(a, b) { return a !== b; });\n});\n\n</script>\n\n\n</body>\n</html>\n, `body` = , `encoded_headers` = {"Content-Type": "text/html; charset=utf-8", "Vary": "Accept-Language, Cookie", "Content-Language": "en"} WHERE `silk_response`.`id` = 4 '
mtford90 commented 10 years ago

Interesting, looks like some kind of encoding issue, see https://docs.djangoproject.com/en/dev/ref/unicode/, in particular the MySQL comments.

I can see from the page content that the charset is utf-8, is your MySQL database configured to support UTF char encodings?

Let me know, if this is the issue i'll add some defensive code to Silk to try and spot this kind of thing. Obviously this error is pretty unhelpful...

Cheers.

mtford90 commented 10 years ago

In the meantime if you set the following in settings.py:

SILKY_MAX_REQUEST_BODY_SIZE = 0
SILKY_MAX_RESPONSE_BODY_SIZE = 0

then silk will stop saving the bodies down. As long as you have no UTF characters in the query params/headers etc the error should go away and you can use Silk minus body inspection

makmanalp commented 10 years ago

That is a good question, you tell me! :) The encodings are set on a per-table basis.

MySQL says of the silk tables:

mysql> show create table silk_response\G
*************************** 1. row ***************************
       Table: silk_response
Create Table: CREATE TABLE `silk_response` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `request_id` int(11) NOT NULL,
  `status_code` int(11) NOT NULL,
  `raw_body` longtext NOT NULL,
  `body` longtext NOT NULL,
  `encoded_headers` longtext NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `request_id` (`request_id`),
  CONSTRAINT `request_id_refs_id_da922f86` FOREIGN KEY (`request_id`) REFERENCES `silk_request` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

mysql> show create table silk_request\G
*************************** 1. row ***************************
       Table: silk_request
Create Table: CREATE TABLE `silk_request` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `path` varchar(300) NOT NULL,
  `query_params` longtext NOT NULL,
  `raw_body` longtext NOT NULL,
  `body` longtext NOT NULL,
  `method` varchar(10) NOT NULL,
  `start_time` datetime NOT NULL,
  `view_name` varchar(300) NOT NULL,
  `end_time` datetime DEFAULT NULL,
  `time_taken` double DEFAULT NULL,
  `encoded_headers` longtext NOT NULL,
  `meta_time` double DEFAULT NULL,
  `meta_num_queries` int(11) DEFAULT NULL,
  `meta_time_spent_queries` double DEFAULT NULL,
  `num_sql_queries` int(11) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `silk_request_73e78684` (`path`),
  KEY `silk_request_b2ae6dce` (`view_name`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

So they're latin-1 by default, it seems.

Thanks for the workaround.

mtford90 commented 10 years ago

Not familiar with MySQL, only used postgres and sqlite which explains why I haven't run into this issue myself.

Did a bit of research and doesn't look like Django provides the ability to set charsets on the tables and will assume UTF-8. See here and here for stackoverflow threads where people are facing a similar issue with MySQL.

Looks like you can use the following for database default charsets:

ALTER DATABASE database_name CHARACTER SET utf8 COLLATE utf8_general_ci;

which should ensure that any tables created by Django take UTF from then on. And then if you want to retroactively fit the existing silk tables:

ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8;

Hope this helps.

mtford90 commented 10 years ago

Closing this as Django does not present any method in which to force table encodings so nothing can be done from the Silk side of things.

medson commented 6 years ago

Not familiar with MySQL, only used postgres and sqlite which explains why I haven't run into this issue myself.

Did a bit of research and doesn't look like Django provides the ability to set charsets on the tables and will assume UTF-8. See here and here for stackoverflow threads where people are facing a similar issue with MySQL.

Looks like you can use the following for database default charsets:

ALTER DATABASE database_name CHARACTER SET utf8 COLLATE utf8_general_ci;

which should ensure that any tables created by Django take UTF from then on. And then if you want to retroactively fit the existing silk tables:

ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8;

Hope this helps.

Thanks, this works for me 😄