publish error: "obj['content'] has unescaped < or > characters!"

[snarfed]

discovered by @jamietanna while trying to publish https://www.jvt.me/mf2/2020/04/qj3zt/ to twitter. thanks jamie! discussion, log:

Params: [('source', 'https://www.jvt.me/mf2/2020/04/qj3zt/'), ('target', 'https://brid.gy/publish/twitter')]
requests.head https://www.jvt.me/mf2/2020/04/qj3zt/ {'headers': {'User-Agent': '...'}}
Source: https://brid.gy/twitter/JamieTanna , features ['publish', 'listen'], status enabled, poll status ok
Publish entity: Publish entity: 'agdicmlkLWd5ckwLEg1QdWJsaXNoZWRQYWdlIiVodHRwczovL3d3dy5qdnQubWUvbWYyLzIwMjAvMDQvcWozenQvDAsSB1B1Ymxpc2gYgIDgza7CmwoM'
requests.get https://www.jvt.me/mf2/2020/04/qj3zt/ {'headers': {'User-Agent': '...'}}
Parsed microformats2: {
  "items":[{
    "type":["h-entry"],
    "properties":{
      "content":[{
        "html":"<p>I'm very much looking forward to this talk, it was awesome when I saw it at <a href=\"/tags/oggcamp/\">#oggcamp</a> last year, and <a href=\"/tags/wit-notts/\">#WiTNotts</a> is very fortunate to have <span class=\"h-card\"><a class=\"u-url\" href=\"https://sparkleclass.com/\">Rachel Morgan-Trimmer</a></span> speak!</p>",
        "value":"I'm very much looking forward to this talk, it was awesome when I saw it at #oggcamp last year, and #WiTNotts is very fortunate to have Rachel Morgan-Trimmer speak!"
      }],
      "bridgy-twitter-content":["I'm very much looking forward to this talk, it was awesome when I saw it at #oggcamp last year, and #WiTNotts is very fortunate to have @SparkleClass speak!"],
      "quotation-of":[{
        "type":["h-cite"],
        "properties":{
          "url":[
            "https://twitter.com/SparkleClass/status/1252949326629933059",
            "https://twitter.com/SparkleClass/status/1252949326629933059"
          ],
          "content":[{
            "html":"<div style=\"white-space:pre\">Today I am mostly working on revising my talk \"The Power of Change: Learning to Live as a Weirdo\" so it works better as an online presentation.\nSign up here: <a href=\"https://www.meetup.com/Women-In-Tech-Nottingham/events/268711710/\">meetup.com/Women-In-Tech-\u2026</a>\nOn Zoom, obvs. For <a href=\"https://twitter.com/WiT_Notts\">@WiT_Notts</a> on 7 May.</div>",
            "value":"Today I am mostly working on revising my talk \"The Power of Change: Learning to Live as a Weirdo\" so it works better as an online presentation. Sign up here: meetup.com/Women-In-Tech-\u2026 On Zoom, obvs. For @WiT_Notts on 7 May."
          }]
        },
        "value":"https://twitter.com/SparkleClass/status/1252949326629933059"
      }],
      "..."
    }
  }],
}
Converted to ActivityStreams object: {
  "id":"https://www.jvt.me/mf2/2020/04/qj3zt/",
  "objectType":"note",
  "published":"2020-04-22T14:51:00+01:00",
  "content":"I'm very much looking forward to this talk, it was awesome when I saw it at #oggcamp last year, and #WiTNotts is very fortunate to have @SparkleClass speak!",
  "url":"https://www.jvt.me/mf2/2020/04/qj3zt/",
  "tags":[{
      "objectType":"hashtag",
      "displayName":"oggcamp"
    },{
      "objectType":"hashtag",
      "displayName":"wit-notts"
    }],
  "attachments":[{
      "objectType":"note",
      "content":"<div style=\"white-space:pre\">Today I am mostly working on revising my talk \"The Power of Change: Learning to Live as a Weirdo\" so it works better as an online presentation.\nSign up here: <a href=\"https://www.meetup.com/Women-In-Tech-Nottingham/events/268711710/\">meetup.com/Women-In-Tech-\u2026</a>\nOn Zoom, obvs. For <a href=\"https://twitter.com/WiT_Notts\">@WiT_Notts</a> on 7 May.</div>",
      "url":"https://twitter.com/SparkleClass/status/1252949326629933059",
      "urls":[
        {"value":"https://twitter.com/SparkleClass/status/1252949326629933059"},
        {"value":"https://twitter.com/SparkleClass/status/1252949326629933059"}
      ]
    }
  ],
  "...
}
Error:  obj['content'] has unescaped < or > characters!

[jamietanna]

Any luck with this, @snarfed?

[snarfed]

haven't looked at it yet, sorry. Real Life™️!

[jamietanna]

No worries - if you're able to give me a hint for how to fix I can try to raise the fix, as unfortunately it's stopping me replying to posts 😥

[snarfed]

yes! the error is here:

https://github.com/snarfed/granary/blob/ee244e2385e1248b644c05cf8e3e541423ccae43/granary/source.py#L822-L823

probably called from here:

https://github.com/snarfed/granary/blob/ee244e2385e1248b644c05cf8e3e541423ccae43/granary/twitter.py#L697-L701

[snarfed]

btw @jamietanna if this is affecting all of your posts, not just one, then it may be a result of a markup change on your site? this check was added on Bridgy's side in July 2019, for #880, so it's not new.

we'll still try to fix it, obviously! just a heads up in case you can revert anything like that on your side as a quicker workaround.

[jamietanna]

@snarfed it was a change on my side, yeah

Folks who follow my blog's #Microformats feed will now see the context (as an h-cite) that is visible on my site, allowing you to have some context for at least interactions with Twitter, but hopefully other stuff in the future!

I'd prefer not to roll it back, so will try and get the Granary stuff sorted tonight :+1: