Jekyll data pulling from JSON

[inetbiz]

I see you documented HUGO. Can you also include a snippet for pulling in from jekyll data?

[martymcguire]

Hi @inetbiz ! Good question!

I don't currently have a Jekyll site for testing, and don't have the time to set one up to make sure I put up a good working example.

However, I do have an old Jekyll version of my site that I used with Morris, so I have the beginnings of a Jekyll snippet.

Would you perhaps be willing to try it out and validate it?

The principal is the same as with Hugo:

1. get a list of webmention IDs for the current page from the index (e.g. site.data.webmention_io.index[page.url])
2. for each index in that list, get the webmention content by ID to do something with.
   • for my old site i have what feels a little goofy in Jekyll's liquid template language, wich is to create an empty array via a hack of splitting an empty string, then using the | push liquid to push the actual webmention content into that array.

I have it in an partial at _includes/mentions.html and it looks like this:

{% assign mention_ids = site.data.webmention_io.index[include.page.url] %}
{% assign all_mentions = "" | split:"" %}
{% for id in mention_ids %}
  {% assign mention = site.data.webmention_io.mentions[id] %}
  {% assign all_mentions = all_mentions | push: mention %}
{% endfor %}

At that point all_mentions has the list of mentions, in the same data structure described in the README for Hugo.

For my personal templates I also had this bit to sort them by when they were received (published date would have been better but not all webmentions have a published date so this is a compromise). I also group them by type because I collect likes and reacji into facepiles, etc:

{% assign all_mentions = all_mentions | sort: "wm-received" %}
{% assign mentions_by_type = all_mentions | group_by: "wm-property" %}

Let me know if that first snippet works for you, or if it can be updated for clarity/correctness!

[inetbiz]

@martymcguire

{% assign mention_ids = site.data.webmention_io.index[include.page.url] %}
{% assign all_mentions = "" | split:"" %}
{% for id in mention_ids %}
  {% assign mention = site.data.webmention_io.mentions[id] %}
  {% assign all_mentions = all_mentions | push: mention %}
{% endfor %}
{% assign all_mentions = all_mentions | sort: "wm-received" %}
{% assign mentions_by_type = all_mentions | group_by: "wm-property" %}
{% if mentions_by_type.like-of != null %}
<section>
  <header>
    <h2 class="p-name">Social Likes</h2>
    {% for like in mentions_by_type.like-of %}
    {% comment %}
    TODO: Figure out if can get title of like.
    {% endcomment %}
      <a class="u-like-of" href="{{like}}">{{like}}</a>{% if forloop.rindex == 2 %} and {% elsif forloop.last != true %}, {% endif %}
    {% endfor %}
  </section>
{% endif %}

I'm just testing to get data. I'll go back and make a facepile It's not pulling data. Sample JSON:

{"type":"entry","author":{"type":"card","name":"🄳🄴🄽🅅🄴🅁 🄿🅁🄾🄿🄷🄸🅃 🄹🅁. 💻🇺🇸","photo":"https://webmention.io/avatar/pbs.twimg.com/6b02b36a90c1ec317ec915e1cb64aa9de7863b3fb977a0c45fc543a760215bc7.jpg","url":"https://twitter.com/DenverProphitJr"},"url":"https://twitter.com/DenverProphitJr/status/1230514964433113094#favorited-by-502274124","published":null,"wm-received":"2020-10-16T16:20:48Z","wm-id":877906,"wm-source":"https://brid-gy.appspot.com/like/twitter/DenverProphitJr/1230514964433113094/502274124","wm-target":"https://denverprophit.us/technical-seo/using-science-philosophy-organize-semantic-web.html","like-of":"https://denverprophit.us/technical-seo/using-science-philosophy-organize-semantic-web.html","wm-property":"like-of","wm-private":false}

[martymcguire]

hey @inetbiz!

it's worth doing some checks at each step to see what you're getting and how it matches up your expectations.

for example, check the value of mention_ids (e.g. with {{ mention_ids | inspect }}) to make sure it's an array of IDs for this page, check mentions_by_type (and maybe all_mentions) to make sure the data is getting loaded correctly, etc.

[inetbiz]

@martymcguire I'm stumped! jekyll 3.9.0 this line is not working: {% assign mention_ids = site.data.webmentions.index[include.page.url] %} The /_data/webmentions/index.json is getting populated.

{"/technical-seo/using-science-philosophy-organize-semantic-web.html":["9a8bbb247ea9c262f2afec3acd66ea0228b1e4b59d92588af0cb11c221f639cf","652913869da404a84ff4a1dda7788daa18b75b955debeb98aa3a43e96e7b8461","3f95b6d4c9488bbb6b36ae1ccccb448c4c58bdfef906f30d4cd46960c28b0a09","8830817b46138898fd045121e34439774796ad81beeac1e340855e8c465bc5e8","b99c970c8a3b9c35942afbf2894e40fd13d4759c2232be0add1c15beedf4a6e5"]}

{{ mention_ids | inspect }} is a null value.

[martymcguire]

@inetbiz ah! that may mean that the lookup index is not what i expected. check the value of include.page.url to make sure it's the path you expect. depending on how you're implementing this, include.page may not be the var representing your page, or url may not be the right property name to get the path to the page, etc.

[inetbiz]

@martymcguire I know I can print {{page.url}} I don't know how to troubleshoot include.page.url?

[martymcguire]

@inetbiz a-ha! perhaps that's a hint as to the problem. i wrote this example as a Jekyll include that is meant to be included in the larger page template. page.url is likely what you are looking for as the lookup value for the index.

[inetbiz]

@martymcguire {% assign mention_ids = site.data.webmentions.index[page.url] %} error: Liquid Exception: undefined method[]' for nil:NilClass in /_layouts/right-sidebar.html`

[martymcguire]

ahhhh, what is your folder structure for your webmentions?

for example, site.data.webmentions.index would mean you'd have something like:

• ... your other jekyll files ...
• _data/
  • webmentions/
  • index.json
  • mentions/
    • <some long hash>.json
    • etc...

if you're storing your mentions in a different folder in _data you'll need to note that in the path.

[inetbiz]

[martymcguire]

oh that's not what i expected / hoped for. :sweat_smile:

can you inspect site.data.webmentions.index and does that look like anything?

[inetbiz]

@martymcguire I changed structure just as you "expected". I did `{{site.data.webmentions.index | inspect}} and got good results.

{{mention_ids | inspect}} is a nil value, still.

[martymcguire]

ah, okay! getting closer! what is the value of page.url?

[inetbiz]

@martymcguire "/technical-seo/using-science-philosophy-organize-semantic-web.html"

[martymcguire]

hmmmm. i am stumped!

your output from {{site.data.webmentions.index | inspect}} looks like it has the dictionary structure i'd expect. (however the quotes surrounding it in the screenshot made me wonder if maybe it's secretly stringified JSON??).

and page.url looks like it should match that dictionary entry.

does {{ site.data.webmentions.index["/technical-seo/using-science-philosophy-organize-semantic-web.html"] | inspect }} give a result? or the same nilClass error? or something new?

[inetbiz]

@martymcguire positive results! ["9a8bbb247ea9c262f2afec3acd66ea0228b1e4b59d92588af0cb11c221f639cf", "652913869da404a84ff4a1dda7788daa18b75b955debeb98aa3a43e96e7b8461", "3f95b6d4c9488bbb6b36ae1ccccb448c4c58bdfef906f30d4cd46960c28b0a09", "8830817b46138898fd045121e34439774796ad81beeac1e340855e8c465bc5e8", "b99c970c8a3b9c35942afbf2894e40fd13d4759c2232be0add1c15beedf4a6e5"]

So {% assign mention_ids = site.data.webmentions.index[include.page.url] %} cannot pass on that dimension key. {% assign mention_ids = site.data.webmentions.index[page.url] %} throws that error: Liquid Exception: undefined method []' for nil:NilClass in /_layouts/right-sidebar.html

[inetbiz]

I fixed it.

{% capture purl %}{{page.url}}{% endcapture %}
{% assign mention_ids = site.data.webmentions.index[purl] %}

[martymcguire]

@inetbiz glad you have it working! somewhat frustrating that you needed to capture {{page.url}} into a variable before the lookup worked successfully. :thinking:

would you be interested in boiling down what you've got working into a minimal example that could be added to the README?

[inetbiz]

So far I have:

<section class="col-8 col-12-medium">
  <article>
    <header>
      <h3>Web Mentions</h3>
    </header>
{% capture purl %}{{page.url}}{% endcapture %}
{% assign mention_ids = site.data.webmentions.index[purl] %}
{% assign all_mentions = "" | split:"" %}
{% for id in mention_ids %}
  {% assign mention = site.data.webmentions.mentions[id] %}
  {% assign all_mentions = all_mentions | push: mention %}
{% endfor %}
{% assign all_mentions = all_mentions | sort: "wm-received" %}
{% assign items = all_mentions | where: "wm-property", "like-of" %}
{% assign items1 = all_mentions | where: "wm-property", "repost-of" %}
<ul class="horizontal">{% for items_hash in items %}
<li class="horizontal"><a title="{{items_hash.author.name}}" href="{{items_hash.author.url}}" target="_blank"><img class="image thumbnail" src="{{items_hash.author.photo}}" alt="{{items_hash.author.name}}"></a></li>
{% if forloop.last %}</ul><p>{{forloop.length}} Likes</p>{%endif%}
{% endfor %}
</article></section></div>
<div class="row">
<section class="col-8 col-12-medium">
  <article>
    <header>
      <h3>Article Reposts</h3>
    </header>
<ul class="horizontal">{% for items_hash in items1 %}
<li class="horizontal"><a title="{{items_hash.author.name}}" href="{{items_hash.author.url}}" target="_blank"><img class="image thumbnail" src="{{items_hash.author.photo}}" alt="{{items_hash.author.name}}"></a></li>
{% if forloop.last %}</ul><p>{{forloop.length}} reposts</p>{%endif%}
{% endfor %}
</article>
</section>

Mention counts could probably be stored in a var and stats displayed higher in the page.

[inetbiz]

@martymcguire would I need to create a method to micropub for comments recycling some of your code and storing in data? I'm not good with php sanitation.

[martymcguire]

@martymcguire would I need to create a method to micropub for comments recycling some of your code and storing in data? I'm not good with php sanitation.

@inetbiz thanks for the example template!

Can you explain a little more about what you mean by Micropub for comments? the code in Morris is hard-coded to understand the webhook format of webmention.io, which is not actually Micropub (the authentication method and data types are totally different).

If you are looking for a way to accept comments via a form directly on your site, it might be possible to use Morris. To make minimal changes to Morris you'd need a bit of Javascript on the frontend to turn the comment form data into JSON that "looks like" the webmention data from webmention.io. You'd also want to extend Morris to accept an additional authentication method beyond the secret token shared with webmention.io.

Does that answer your question?

[inetbiz]

@martymcguire I bet I could include the login authentication from indieauth. I'll work on that project and ping you. =)

[martymcguire]

@inetbiz authentication is definitely not the only thing to worry about for Micropub!

It's worth looking into some of the dedicated Micropub implementations around for generating content files for static sites.

I use a fork of skpy/Micropub for my own site and I like how it's structured!