search

simonw / til

Today I Learned
https://til.simonwillison.net
Apache License 2.0
1.02k stars 81 forks source link

Host these on til.simonwillison.net #17

Closed simonw closed 3 years ago

simonw commented 3 years ago

Right now I link to the GitHub rendered page, which has a couple of downsides:

I have pages for these here https://til.simonwillison.net/til/til/svg_dynamic-line-chart.md but they don't handle all of GitHub Flavored Markdown

simonw commented 3 years ago

Rather than figuring out how to exactly render GFM in Python, I'm tempted to use the GitHub API to cache a generated rendered version instead.

simonw commented 3 years ago

Hit https://api.github.com/repos/simonw/til/contents/svg/dynamic-line-chart.md with an accept of application/vnd.github.VERSION.html

In [6]: print(httpx.get('https://api.github.com/repos/simonw/til/contents/svg/dynamic-line-chart.md', headers={"Accept": "application/vnd.github.VERSION.html"}).text)
<div id="file" class="md" data-path="svg/dynamic-line-chart.md"><article class="markdown-body entry-content container-lg" itemprop="text"><h1><a id="user-content-creating-a-dynamic-line-chart-with-svg" class="anchor" aria-hidden="true" href="#creating-a-dynamic-line-chart-with-svg"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Creating a dynamic line chart with SVG</h1>
<p>I helped build the tide chart visualizations for <a href="https://www.rockybeaches.com/" rel="nofollow">Rocky Beaches</a>.</p>
<p>I wanted to generate an SVG line representing 24 hours of tide levels. I had 240 points per day of level data, imported from the NOAA Tides &amp; Currents API. That data looked <a href="https://www.rockybeaches.com/data/tide_predictions" rel="nofollow">like this</a>:</p>
<table>
<thead>
<tr>
<th>station_id</th>
<th>datetime</th>
<th>mllw_feet</th>
</tr>
</thead>
<tbody>
<tr>
<td>9414131</td>
<td>2020-08-19 00:00</td>
<td>5.913</td>
</tr>
<tr>
<td>9414131</td>
<td>2020-08-19 00:06</td>
<td>5.822</td>
</tr>
<tr>
<td>9414131</td>
<td>2020-08-19 00:12</td>
<td>5.726</td>
</tr>
<tr>
<td>9414131</td>
<td>2020-08-19 00:18</td>
<td>5.623</td>
</tr>
<tr>
<td>9414131</td>
<td>2020-08-19 ...</td>
<td>...</td>
</tr>
</tbody>
</table>
<p>I started with <a href="https://css-tricks.com/how-to-make-charts-with-svg/#line-charts" rel="nofollow">this line chart example</a> from CSS-Tricks:</p>
<div class="highlight highlight-text-xml-svg"><pre>&lt;<span class="pl-ent">svg</span> <span class="pl-e">viewBox</span>=<span class="pl-s"><span class="pl-pds">"</span>0 0 500 100<span class="pl-pds">"</span></span>&gt;
  &lt;<span class="pl-ent">polyline</span>
     <span class="pl-e">fill</span>=<span class="pl-s"><span class="pl-pds">"</span>none<span class="pl-pds">"</span></span>
     <span class="pl-e">stroke</span>=<span class="pl-s"><span class="pl-pds">"</span>#0074d9<span class="pl-pds">"</span></span>
     <span class="pl-e">stroke-width</span>=<span class="pl-s"><span class="pl-pds">"</span>3<span class="pl-pds">"</span></span>
     <span class="pl-e">points</span>=<span class="pl-s"><span class="pl-pds">"</span></span>
<span class="pl-s">       0,120</span>
<span class="pl-s">       20,60</span>
<span class="pl-s">       40,80</span>
<span class="pl-s">       60,20<span class="pl-pds">"</span></span>/&gt;
&lt;/<span class="pl-ent">svg</span>&gt;</pre></div>
<p>A few things to note:</p>
<ul>
<li>The SVG coordinate system starts at the top left - so '0,120' means 0 units along, 120 units down.</li>
<li>The <code>viewBox</code> attribute specifies <code>min-x min-y width height</code> - so <code>0 0 500 100</code> specifies a box that is 500 units wide and 100 units high.</li>
</ul>
<p>The dynamic data can be contained entirely in that single <code>points=</code> attribute on the <code>polyline</code>.</p>
<h2><a id="user-content-prototyping-using-sql" class="anchor" aria-hidden="true" href="#prototyping-using-sql"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Prototyping using SQL</h2>
<p>I posted my working <a href="https://github.com/natbat/rockybeaches/issues/31">in this issue</a>, including prototyping the polyline using a SQL query that generated the <code>x1,y1 x2,y2</code> pairs. That query looked like this:</p>
<div class="highlight highlight-source-sql"><pre>with today_points <span class="pl-k">as</span> (
  <span class="pl-k">select</span>
    datetime,
    mllw_feet
  <span class="pl-k">from</span>
    tide_predictions
  <span class="pl-k">where</span>
    <span class="pl-k">date</span>(<span class="pl-s"><span class="pl-pds">"</span>datetime<span class="pl-pds">"</span></span>) <span class="pl-k">=</span> :p0
    <span class="pl-k">and</span> <span class="pl-s"><span class="pl-pds">"</span>station_id<span class="pl-pds">"</span></span> <span class="pl-k">=</span> :p1
),
min_max <span class="pl-k">as</span> (
  <span class="pl-k">select</span>
    <span class="pl-c1">min</span>(mllw_feet) <span class="pl-k">as</span> min_feet,
    <span class="pl-c1">max</span>(mllw_feet) <span class="pl-k">as</span> max_feet
  <span class="pl-k">from</span>
    today_points
),
points <span class="pl-k">as</span> (
  <span class="pl-k">select</span>
    RANK () OVER (
      <span class="pl-k">ORDER BY</span>
        datetime
    ) <span class="pl-k">-</span><span class="pl-c1">1</span> <span class="pl-k">as</span> rank,
    <span class="pl-c1">min_max</span>.<span class="pl-c1">min_feet</span>,
    <span class="pl-c1">min_max</span>.<span class="pl-c1">max_feet</span>,
    mllw_feet,
    (
      <span class="pl-c1">100</span> <span class="pl-k">*</span> (mllw_feet <span class="pl-k">-</span> <span class="pl-c1">min_max</span>.<span class="pl-c1">min_feet</span>) <span class="pl-k">/</span> (<span class="pl-c1">min_max</span>.<span class="pl-c1">max_feet</span> <span class="pl-k">-</span> <span class="pl-c1">min_max</span>.<span class="pl-c1">min_feet</span>)
    ) <span class="pl-k">as</span> line_height_pct
  <span class="pl-k">from</span>
    tide_predictions,
    min_max
  <span class="pl-k">where</span>
    <span class="pl-k">date</span>(<span class="pl-s"><span class="pl-pds">"</span>datetime<span class="pl-pds">"</span></span>) <span class="pl-k">=</span> :p0
    <span class="pl-k">and</span> <span class="pl-s"><span class="pl-pds">"</span>station_id<span class="pl-pds">"</span></span> <span class="pl-k">=</span> :p1
  <span class="pl-k">order by</span>
    datetime
)
<span class="pl-k">select</span>
  group_concat(rank <span class="pl-k">||</span> <span class="pl-s"><span class="pl-pds">'</span>,<span class="pl-pds">'</span></span> <span class="pl-k">||</span> line_height_pct, <span class="pl-s"><span class="pl-pds">'</span> <span class="pl-pds">'</span></span>)
<span class="pl-k">from</span>
  points</pre></div>
<p><a href="https://www.rockybeaches.com/data?sql=with+today_points+as+%28%0D%0A++select+datetime%2C+mllw_feet%0D%0A++from+tide_predictions%0D%0A++where%0D%0A++date%28%22datetime%22%29+%3D+%3Ap0%0D%0A++and+%22station_id%22+%3D+%3Ap1%0D%0A%29%2C+min_max+as+%28select+min%28mllw_feet%29+as+min_feet%2C+max%28mllw_feet%29+as+max_feet+from+today_points%29%2C+points+as+%28select%0D%0A++RANK+%28%29+OVER+%28%0D%0A++++ORDER+BY%0D%0A++++++datetime%0D%0A++%29+-1+as+rank%2C%0D%0A+min_max.min_feet%2C+min_max.max_feet%2C++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++mllw_feet%2C%0D%0A++%28100+*+%28mllw_feet+-+min_max.min_feet%29+%2F+%28min_max.max_feet+-+min_max.min_feet%29%29+as+line_height_pct%0D%0Afrom%0D%0A++tide_predictions%2C+min_max%0D%0Awhere%0D%0A++date%28%22datetime%22%29+%3D+%3Ap0%0D%0A++and+%22station_id%22+%3D+%3Ap1%0D%0Aorder+by%0D%0A++datetime%29%0D%0Aselect+group_concat%28rank+%7C%7C+%27%2C%27+%7C%7C+line_height_pct%2C+%27+%27%29+from+points&amp;p0=2020-08-21&amp;p1=9414131" rel="nofollow">Try that here</a></p>
<p>I then pasted the results into the SVG and previewed it by pasting it into <a href="https://htmledit.squarefree.com/" rel="nofollow">https://htmledit.squarefree.com/</a></p>
<div class="highlight highlight-text-html-basic"><pre><span class="pl-kos">&lt;</span><span class="pl-ent">svg</span> <span class="pl-c1">style</span>="<span class="pl-s">border: 1px solid red; width: 40%; height: 60px</span>" <span class="pl-c1">viewBox</span>="<span class="pl-s">0 -2 240 104</span>" <span class="pl-c1">preserveAspectRatio</span>="<span class="pl-s">none</span>"<span class="pl-kos">&gt;</span>
  <span class="pl-kos">&lt;</span><span class="pl-ent">polyline</span>
     <span class="pl-c1">fill</span>="<span class="pl-s">none</span>"
     <span class="pl-c1">stroke</span>="<span class="pl-s">#0074d9</span>"
     <span class="pl-c1">stroke-width</span>="<span class="pl-s">2</span>"
     <span class="pl-c1">points</span>="<span class="pl-s">0,0.543825975687781 1,0.255918106206011 2,0.0799744081893721 3,1.4210854715202e-14 4,0.0159948816378659 5,0.159948816378744 6,0.383877159309023 7,0.71976967370442 8,1.16762635956493 9,1.71145233525272 10,2.36724248240563 11,3.11900191938578 12,3.98272552783109 13,4.94241842610363 14,5.99808061420346 15,7.16570697376839 16,8.4133077415227 17,9.75687779910427 18,11.1804222648752 19,12.6999360204734 20,14.3154190658989 21,15.9948816378759 22,17.7543186180422 23,19.5937300063979 24,21.4971209213052 25,23.4644913627639 26,25.4958413307741 27,27.5911708253359 28,29.7344849648113 29,31.9097888675624 30,34.149072296865 31,36.4203454894434 32,38.7236084452975 33,41.0428662827895 34,43.3941138835573 35,45.7613563659629 36,48.1445937300064 37,50.5278310940499 38,52.9270633397313 39,55.3103007037748 40,57.6775431861804 41,60.0287907869482 42,62.3640435060781 43,64.6673064619322 44,66.9385796545106 45,69.1618682021753 46,71.3371721049264 47,73.4804862444018 48,75.5598208573257 49,77.575175943698 50,79.5265515035189 51,81.4139475367882 52,83.2213691618682 53,84.9488163787588 54,86.59628918746 55,88.1637875879719 56,89.6513115802943 57,91.0268714011516 58,92.3224568138196 59,93.5220729366603 60,94.6257197696737 61,95.6333973128599 62,96.5291106845809 63,97.3288547664747 64,98.0326295585413 65,98.6244401791427 66,99.104286628279 67,99.488163787588 68,99.7600767754319 69,99.9360204734485 70,100.0 71,99.9520153550864 72,99.7920665387076 73,99.5361484325016 74,99.1682661548304 75,98.6884197056942 76,98.0966090850928 77,97.4088291746641 78,96.6250799744082 79,95.7293666026871 80,94.7376839411388 81,93.6340371081254 82,92.4344209852847 83,91.1548304542546 84,89.7792706333973 85,88.3077415227127 86,86.7562380038388 87,85.1247600767754 88,83.4133077415227 89,81.6218809980806 90,79.7824696097249 91,77.8630838131798 92,75.9117082533589 93,73.8963531669866 94,71.8330134357006 95,69.7376839411388 96,67.5943698016635 97,65.4350607805502 98,63.2597568777991 99,61.0684580934101 100,58.8611644273832 101,56.6538707613564 102,54.4625719769674 103,52.2552783109405 104,50.0799744081894 105,47.9206653870761 106,45.7773512476008 107,43.6660268714011 108,41.5866922584773 109,39.5393474088292 110,37.5399872040947 111,35.5886116442738 112,33.6692258477287 113,31.829814459373 114,30.022392834293 115,28.2949456174024 116,26.6154830454255 117,25.0159948816379 118,23.4804862444018 119,22.0089571337172 120,20.6333973128599 121,19.3218170185541 122,18.1062060140755 123,16.9705694177863 124,15.9149072296865 125,14.9552143314139 126,14.0914907229686 127,13.3237364043506 128,12.635956493922 129,12.0601407549584 130,11.5802943058221 131,11.212412028151 132,10.9245041586692 133,10.7485604606526 134,10.6685860524632 135,10.7005758157389 136,10.828534868842 137,11.0524632117722 138,11.3723608445297 139,11.7882277671145 140,12.3000639795265 141,12.9078694817658 142,13.5956493921945 143,14.3793985924504 144,15.2431222008957 145,16.2028150991683 146,17.2264875239923 147,18.3301343570058 148,19.4977607165707 149,20.745361484325 150,22.0409468969929 151,23.4005118362124 152,24.8240563019834 153,26.2955854126679 154,27.7991042866283 155,29.3666026871401 156,30.9660908509277 157,32.5815738963532 158,34.2450415866923 159,35.9245041586692 160,37.6199616122841 161,39.3474088291747 162,41.0588611644274 163,42.786308381318 164,44.5137555982086 165,46.2412028150992 166,47.9526551503519 167,49.6481126039667 168,51.3275751759437 169,52.9750479846449 170,54.5905310300704 171,56.1740243122201 172,57.725527831094 173,59.2130518234165 174,60.6685860524632 175,62.0761356365963 176,63.40371081254 177,64.6992962252079 178,65.9149072296865 179,67.0665387076136 180,68.1541906589891 181,69.1618682021753 182,70.10556621881 183,70.9692898272553 184,71.7530390275112 185,72.4408189379399 186,73.064619321817 187,73.6084452975048 188,74.0563019833653 189,74.4241842610365 190,74.7120921305182 191,74.9040307101727 192,75.0159948816379 193,75.0319897632758 194,74.9520153550864 195,74.8080614203455 196,74.5521433141395 197,74.2162507997441 198,73.8003838771593 199,73.2885476647473 200,72.680742162508 201,72.0089571337172 202,71.2412028150992 203,70.3774792066539 204,69.4497760716571 205,68.4420985284709 206,67.3544465770953 207,66.1868202175304 208,64.9552143314139 209,63.6436340371081 210,62.2840690978887 211,60.8445297504799 212,59.3730006397953 213,57.8214971209213 214,56.2380038387716 215,54.6225207933461 216,52.9430582213691 217,51.2476007677543 218,49.5201535508637 219,47.7767114523352 220,46.0172744721689 221,44.2418426103647 222,42.4664107485605 223,40.6749840051184 224,38.8995521433141 225,37.1401151631478 226,35.3806781829814 227,33.6532309660909 228,31.9417786308381 229,30.2623160588612 230,28.61484325016 231,26.9993602047345 232,25.4318618042226 233,23.9123480486244 234,22.424824056302 235,21.001279590531 236,19.6257197696737 237,18.3141394753679 238,17.0665387076136 239,15.8829174664107</span>"/&gt;
<span class="pl-kos">&lt;/</span><span class="pl-ent">svg</span><span class="pl-kos">&gt;</span></pre></div>
<h2><a id="user-content-scaling-the-image-with-preserveaspectrationone" class="anchor" aria-hidden="true" href="#scaling-the-image-with-preserveaspectrationone"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Scaling the image with preserveAspectRatio="none"</h2>
<p>I wanted to scale the SVG image to fit the div it was overlayed onto - which had a fixed height of 60px but a variable width.</p>
<p>SVG images maintain their aspect ratio by default. Adding <code>preserveAspectRatio="none"</code> changed this, so the image could be distorted to fit the shape.</p>
<p>Unfortunately this has a visual impact on the line of the graph - causing it to be wider and narrower at different angles. We decided that this brush-stroke effect was actually OK for our purposes.</p>
<p><a target="_blank" rel="noopener noreferrer" href="https://user-images.githubusercontent.com/9599/90821264-7a930680-e2e7-11ea-83a2-3003cf08877f.png"><img src="https://user-images.githubusercontent.com/9599/90821264-7a930680-e2e7-11ea-83a2-3003cf08877f.png" alt="Distorted line" style="max-width:100%;"></a></p>
<h2><a id="user-content-tweaking-the-viewbox" class="anchor" aria-hidden="true" href="#tweaking-the-viewbox"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Tweaking the viewBox</h2>
<p>My chart was drawn be 240 units wide (covering 24 hours of the day with 10 units per hour) and 100 units high.</p>
<p>Using this as the exact viewBox dimensions caused the top and bottom of the line peaks to be truncated. I fixed that by tweaking the viewBox a bit to add some breathing room:</p>
<pre><code>viewBox="0 -2 240 104"
</code></pre>
<h2><a id="user-content-generating-the-data-using-python" class="anchor" aria-hidden="true" href="#generating-the-data-using-python"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Generating the data using Python</h2>
<p>Here's <a href="https://github.com/natbat/rockybeaches/blob/70039f18b3d3823a4f069deca513e950a3aaba4f/plugins/template_vars.py#L148-L156">the final Python code</a> I used to generate the points:</p>
<div class="highlight highlight-source-python"><pre><span class="pl-c"># Calculate SVG points, refs https://github.com/natbat/rockybeaches/issues/31</span>
<span class="pl-s1">min_feet</span> <span class="pl-c1">=</span> <span class="pl-en">min</span>(<span class="pl-s1">h</span>[<span class="pl-s">"feet"</span>] <span class="pl-k">for</span> <span class="pl-s1">h</span> <span class="pl-c1">in</span> <span class="pl-s1">heights</span>[<span class="pl-c1">1</span>:<span class="pl-c1">-</span><span class="pl-c1">1</span>])
<span class="pl-s1">max_feet</span> <span class="pl-c1">=</span> <span class="pl-en">max</span>(<span class="pl-s1">h</span>[<span class="pl-s">"feet"</span>] <span class="pl-k">for</span> <span class="pl-s1">h</span> <span class="pl-c1">in</span> <span class="pl-s1">heights</span>[<span class="pl-c1">1</span>:<span class="pl-c1">-</span><span class="pl-c1">1</span>])
<span class="pl-s1">feet_delta</span> <span class="pl-c1">=</span> <span class="pl-s1">max_feet</span> <span class="pl-c1">-</span> <span class="pl-s1">min_feet</span>
<span class="pl-s1">svg_points</span> <span class="pl-c1">=</span> []
<span class="pl-k">for</span> <span class="pl-s1">i</span>, <span class="pl-s1">height</span> <span class="pl-c1">in</span> <span class="pl-en">enumerate</span>(<span class="pl-s1">heights</span>[<span class="pl-c1">1</span>:<span class="pl-c1">-</span><span class="pl-c1">1</span>]):
    <span class="pl-s1">ratio</span> <span class="pl-c1">=</span> (<span class="pl-s1">height</span>[<span class="pl-s">"feet"</span>] <span class="pl-c1">-</span> <span class="pl-s1">min_feet</span>) <span class="pl-c1">/</span> <span class="pl-s1">feet_delta</span>
    <span class="pl-s1">line_height_pct</span> <span class="pl-c1">=</span> <span class="pl-c1">100</span> <span class="pl-c1">-</span> (<span class="pl-s1">ratio</span> <span class="pl-c1">*</span> <span class="pl-c1">100</span>)
    <span class="pl-s1">svg_points</span>.<span class="pl-en">append</span>((<span class="pl-s1">i</span>, <span class="pl-s1">line_height_pct</span>))
<span class="pl-c"># ...</span>
<span class="pl-s1">points</span> <span class="pl-c1">=</span> <span class="pl-s">" "</span>.<span class="pl-en">join</span>(<span class="pl-s">"{},{:.2f}"</span>.<span class="pl-en">format</span>(<span class="pl-s1">i</span>, <span class="pl-s1">pct</span>) <span class="pl-k">for</span> <span class="pl-s1">i</span>, <span class="pl-s1">pct</span> <span class="pl-c1">in</span> <span class="pl-s1">svg_points</span>)</pre></div>
<p>I used <code>{:.2f}</code> to truncate the floating point represention to just two decimal places, which knocked 100KB off the total page size due to the number of charts being displayed!</p>
<p>Finished result can be seen here: <a href="https://www.rockybeaches.com/us/pillar-point#when-to-visit" rel="nofollow">https://www.rockybeaches.com/us/pillar-point#when-to-visit</a></p>
</article></div>
simonw commented 3 years ago

To avoid hitting that API more often than necessary, I can store the content in a column and have another column showing the hash of that value. If the hash has changed I can re-fetch the rendered version.

simonw commented 3 years ago

Even better:

In [10]: httpx.post("https://api.github.com/markdown", json={
    ...:   "mode": "gfm",
    ...:   "text": "# Hello"
    ...:   }).text
Out[10]: '<h1>Hello</h1>'
simonw commented 3 years ago

I don't even need to calculate a hash - I can do an exact comparison of the body column to see if the markdown changed.

simonw commented 3 years ago

The headers reveal details of the rate limits:

{'server': 'GitHub.com',
 'date': 'Sat, 22 Aug 2020 19:20:29 GMT',
 'content-type': 'text/html;charset=utf-8',
 'content-length': '14',
 'status': '200 OK',
 'x-commonmarker-version': '0.21.0',
 'x-ratelimit-limit': '60',
 'x-ratelimit-remaining': '57',
 'x-ratelimit-reset': '1598125911',
 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
 'access-control-allow-origin': '*',
 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
 'x-frame-options': 'deny',
 'x-content-type-options': 'nosniff',
 'x-xss-protection': '1; mode=block',
 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
 'content-security-policy': "default-src 'none'",
 'vary': 'Accept-Encoding, Accept, X-Requested-With',
 'x-github-request-id': 'C9F0:12D7:32CDD4:8064E2:5F416FFD'}
simonw commented 3 years ago

This also means I can fix the Atom feed in #12.

simonw commented 3 years ago

https://til.simonwillison.net/til/til/svg_dynamic-line-chart.md is rendering correctly now!