search

jackjamieson2 / yarns-microsub-server

Yarns is a Microsub server that runs as a plugin on your WordPress site.
GNU General Public License v2.0
20 stars 4 forks source link

Timeline issues in Together #89

Closed jackjamieson2 closed 4 years ago

jackjamieson2 commented 5 years ago

I haven't troubleshooted this properly yet, but have a rough sense.

Together (alltogethernow.io) has somewhat strict expectations for how timeline items should be structured. When a field in a timeline item is in the wrong format, Together will not display the timeline for its channel at all.

Fortunately Together's error reporting is thorough so I should be able to work through these issues.

I'll collect issues here and try to address them together.

  1. Timeline's containing twitter posts (fetched through granary)
    • It seems like the main issue here is with the category field.
jackjamieson2 commented 5 years ago

Re: granary items

Found an offending item from Twitter. I fetched this using granary.io to present my twitter timeline as html with microformats.

When trying to display a timeline containing this item, Together returns the error: Expected Iterable, but did not find one for field Post.category.

Here's the HTML generated by granary:

<article class="u-repost-of h-cite">
  <span class="p-uid">tag:twitter.com:1155664307058880512</span>

  <time class="dt-published" datetime="2019-07-29T02:20:16+00:00">2019-07-29T02:20:16+00:00</time>

  <span class="p-author h-card">
    <data class="p-uid" value="tag:twitter.com:SMLabTO"></data>
<data class="p-numeric-id" value="194159907"></data>
    <a class="p-name u-url" href="https://twitter.com/SMLabTO">Social Media Lab</a>
<a class="u-url" href="http://socialmedialab.ca/"></a>
    <span class="p-nickname">SMLabTO</span>
    <img class="u-photo" src="https://pbs.twimg.com/profile_images/639293371144474629/ORfuBbRd.jpg" alt="">
  </span>

  <a class="u-url" href="https://twitter.com/SMLabTO/status/1155664307058880512">https://twitter.com/SMLabTO/status/1155664307058880512</a>
  <div class="e-content p-name">

  Hi <a href="https://twitter.com/search?q=%23SMSociety">#SMSociety</a>... If you are using Twitter data in your research, this thread might be of interest. Also consider completing the survey from <a href="https://twitter.com/TwitterDev">@TwitterDev</a>. The link is in the thread.
  </div>

  <span class="u-category h-card">
    <data class="p-uid" value="tag:twitter.com:TwitterDev"></data>
    <a class="p-name u-url" href="https://twitter.com/TwitterDev">Twitter Dev</a> 
  </span>

<article class="u-quotation-of h-cite">
  <span class="p-uid">tag:twitter.com:1151520361529430016</span>

  <time class="dt-published" datetime="2019-07-17T15:53:43+00:00">2019-07-17T15:53:43+00:00</time>

  <span class="p-author h-card">
    <data class="p-uid" value="tag:twitter.com:TwitterDev"></data>
<data class="p-numeric-id" value="2244994945"></data>
    <a class="p-name u-url" href="https://twitter.com/TwitterDev">Twitter Dev</a>
<a class="u-url" href="https://developer.twitter.com/"></a>
<a class="u-url" href="https://twittercommunity.com/"></a>
    <span class="p-nickname">TwitterDev</span>
    <img class="u-photo" src="https://pbs.twimg.com/profile_images/880136122604507136/xHrnqf1T.jpg" alt="">
  </span>

  <a class="u-url" href="https://twitter.com/TwitterDev/status/1151520361529430016">https://twitter.com/TwitterDev/status/1151520361529430016</a>
  <div class="e-content p-name">

  Academic research is some of the most impactful work that happens with the Twitter API. As we plan for the future of our developer platform we want to learn more from this community. (Thread👇)
  </div>

</article>

</article>

Yarns converts this into the following jf2:

 {
                "category":
                {
                    "uid": "tag:twitter.com:TwitterDev",
                    "name": "Twitter Dev",
                    "url": "https://twitter.com/TwitterDev",
                    "type": "card"
                },
                "repost-of":
                [
                    "https://twitter.com/SMLabTO/status/1155664307058880512"
                ],
                "type": "entry",
                "url": "https://twitter.com/ligova/status/1156295567271759879",
                "published": "2019-07-30T20:08:40+00:00",
                "updated": "2019-07-30T20:08:40+00:00",
                "author":
                {
                    "name": "Lina Gomez",
                    "type": "card",
                    "uid": "tag:twitter.com:ligova",
                    "numeric-id":
                    [
                        "162495567"
                    ],
                    "nickname":
                    [
                        "ligova"
                    ],
                    "url": "https://twitter.com/ligova",
                    "photo": "https://pbs.twimg.com/profile_images/1135634387687620608/Lryi9Jd6.jpg"
                },
                "content":
                {
                    "text": "RT @SMLabTO Hi #SMSociety... If you are using Twitter data in your research, this thread might be of interest. Also consider completing the survey from @TwitterDev. The link is in the thread.",
                    "html": "RT <a href=\"https://twitter.com/SMLabTO/status/1155664307058880512\">@SMLabTO</a> Hi <a href=\"https://twitter.com/search?q=%23SMSociety\">#SMSociety</a>... If you are using Twitter data in your research, this thread might be of interest. Also consider completing the survey from <a href=\"https://twitter.com/TwitterDev\">@TwitterDev</a>. The link is in the thread."
                },
                "references":
                {
                    "https://twitter.com/SMLabTO/status/1155664307058880512":
                    {
                        "category":
                        {
                            "uid": "tag:twitter.com:TwitterDev",
                            "name": "Twitter Dev",
                            "url": "https://twitter.com/TwitterDev",
                            "type": "card"
                        },
                        "quotation-of":
                        {
                            "type": "cite",
                            "url": "https://twitter.com/TwitterDev/status/1151520361529430016",
                            "published": "2019-07-17T15:53:43+00:00",
                            "updated": "2019-07-17T15:53:43+00:00",
                            "author":
                            {
                                "uid": "tag:twitter.com:TwitterDev",
                                "numeric-id":
                                [
                                    "2244994945"
                                ],
                                "name": "Twitter Dev",
                                "nickname":
                                [
                                    "TwitterDev"
                                ],
                                "url": "https://twitter.com/TwitterDev",
                                "photo": "https://pbs.twimg.com/profile_images/880136122604507136/xHrnqf1T.jpg",
                                "type": "card"
                            },
                            "content":
                            {
                                "text": "Academic research is some of the most impactful work that happens with the Twitter API. As we plan for the future of our developer platform we want to learn more from this community. (Thread👇)",
                                "html": "Academic research is some of the most impactful work that happens with the Twitter API. As we plan for the future of our developer platform we want to learn more from this community. (Thread👇)"
                            }
                        },
                        "type": "cite",
                        "url": "https://twitter.com/SMLabTO/status/1155664307058880512",
                        "published": "2019-07-29T02:20:16+00:00",
                        "updated": "2019-07-29T02:20:16+00:00",
                        "author":
                        {
                            "uid": "tag:twitter.com:SMLabTO",
                            "numeric-id":
                            [
                                "194159907"
                            ],
                            "name": "Social Media Lab",
                            "nickname":
                            [
                                "SMLabTO"
                            ],
                            "url": "https://twitter.com/SMLabTO",
                            "photo": "https://pbs.twimg.com/profile_images/639293371144474629/ORfuBbRd.jpg",
                            "type": "card"
                        },
                        "content":
                        {
                            "text": "Hi #SMSociety... If you are using Twitter data in your research, this thread might be of interest. Also consider completing the survey from @TwitterDev. The link is in the thread.",
                            "html": "Hi <a href=\"https://twitter.com/search?q=%23SMSociety\">#SMSociety</a>... If you are using Twitter data in your research, this thread might be of interest. Also consider completing the survey from <a href=\"https://twitter.com/TwitterDev\">@TwitterDev</a>. The link is in the thread."
                        }
                    }
                },
                "post-type": "repost",
                "date": "2019-07-30T20:08:40+00:00",
                "_id": 15309,
                "_is_read": false
            },

From my perspective, I don't understand why Granary creates the following category:

  <span class="u-category h-card">
    <data class="p-uid" value="tag:twitter.com:TwitterDev"></data>
    <a class="p-name u-url" href="https://twitter.com/TwitterDev">Twitter Dev</a> 
  </span>

Will look into this further and think about what to do.

Is granary converting this oddly? If so, open an issue for granary. If not, then add some cleaning function to validate category fields returned by Yarns

jackjamieson2 commented 5 years ago

This will be addressed in https://github.com/dshanske/parse-this. Solution is to normalize category so it only contains strings. The references property can be used to store the additional h-card info for clients that want to use it.

Started writing a PR for Parse-This. So far it works if there is only one category, but I should revise to account for multiple categories

if ( isset( $item['category'] ) ) {
    if ( isset( $item['category']['type'] ) ) {
        if ( 'card' === $item['category']['type'] ) {
            // Category is an h-card
            if ( isset( $item['category']['url'] ) ) {
                $ref_id = $item['category']['url'];
            } elseif ( isset( $item['category']['uid'] ) ) {
                $ref_id = $item['category']['uid'];
            } elseif ( isset( $item['category']['name'] ) ) {
                $ref_id = $item['category']['name'];
            }

            if ( ! empty( $ref_id ) ) {
                if ( ! isset( $item['references'] ) ) {
                    $item['references'] = [];
                }

                if (!isset($item['references'][$ref_id])) {
                    $item['references'][$ref_id] = $item['category'];
                }

                $item['category'] = $ref_id;
            }
            Yarns_MicroSub_Plugin::debug_log( 'category stuff: ' . wp_json_encode( $item ) );
        }
    }
}