kbravh / tweet-to-markdown

A command line tool to convert Tweets to Markdown.
MIT License
159 stars 9 forks source link

Possible @mention parsing bug #9

Closed gapmiss closed 2 years ago

gapmiss commented 2 years ago

Possible edge case. Version 2.1.1 ttm does not appear to parse the @mentions in this tweet correctly.

https://twitter.com/jose_edil/status/1538271708918034433

This is the resulting md:

Notice the double square brackets following the 1st @mention. Also, viewing the original tweet, "@Its_Prasa" is not a current twitter.com user.

---
author: "Edil Medeiros 🚒 βœοΈπŸ΄β€β˜ οΈ 🧩"
handle: "@jose_edil"
source: "https://twitter.com/jose_edil/status/1538271708918034433"
likes: 14
retweets: 0
replies: 4
---
![jose_edil](https://pbs.twimg.com/profile_images/669315274353561600/eHheoab4_normal.jpg)
Edil Medeiros 🚒 βœοΈπŸ΄β€β˜ οΈ 🧩 ([@jose_edil](https://twitter.com/jose_edil))

[@visualizevalue](https://twitter.com/visualizevalue) [[@EvansNifty](https://twitter.com/EvansNifty)](https://twitter.com/EvansNifty) [[@OzolinsJanis](https://twitter.com/OzolinsJanis)](https://twitter.com/OzolinsJanis) [[@milanicreative](https://twitter.com/milanicreative)](https://twitter.com/milanicreative) [[@design_by_kp](https://twitter.com/design_by_kp)](https://twitter.com/design_by_kp) [[@victor_bigfield](https://twitter.com/victor_bigfield)](https://twitter.com/victor_bigfield) [[@StartupIllustr](https://twitter.com/StartupIllustr)](https://twitter.com/StartupIllustr) [[@tracytangtt](https://twitter.com/tracytangtt)](https://twitter.com/tracytangtt) [[@AlexMaeseJ](https://twitter.com/AlexMaeseJ)](https://twitter.com/AlexMaeseJ) [[@ash_lmb](https://twitter.com/ash_lmb)](https://twitter.com/ash_lmb) [[@moina_abdul](https://twitter.com/moina_abdul)](https://twitter.com/moina_abdul) @Its_Prasa [[@elliottaleksndr](https://twitter.com/elliottaleksndr)](https://twitter.com/elliottaleksndr) [[@aaraalto](https://twitter.com/aaraalto)](https://twitter.com/aaraalto) [[@tanoseihito](https://twitter.com/tanoseihito)](https://twitter.com/tanoseihito) [[@jeffkortenbosch](https://twitter.com/jeffkortenbosch)](https://twitter.com/jeffkortenbosch) [[@FerraroRoberto](https://twitter.com/FerraroRoberto)](https://twitter.com/FerraroRoberto) [[@eneskartall](https://twitter.com/eneskartall)](https://twitter.com/eneskartall) [[@SachinRamje](https://twitter.com/SachinRamje)](https://twitter.com/SachinRamje) [[@AidanYeep](https://twitter.com/AidanYeep)](https://twitter.com/AidanYeep) [[@jozzua](https://twitter.com/jozzua)](https://twitter.com/jozzua) Here they are:

@EvansNifty
@OzolinsJanis
@milanicreative
@design_by_kp
@victor_bigfield
@StartupIllustr
@tracytangtt
@AlexMaeseJ
@ash_lmb
@moina_abdul
@Its_Prasa
@elliottaleksndr
@aaraalto
@tanoseihito
@jeffkortenbosch
@FerraroRoberto
@eneskartall
@SachinRamje
@AidanYeep
@jozzua

[Tweet link](https://twitter.com/jose_edil/status/1538271708918034433)

I've tested w/ single tweet, thread, and condensed_thread.

ttm -b '<TWITTER_BEARER_TOKEN>' --filename '[[handle]]-[[id]]' --path 'markdown/' -fqm https://twitter.com/i/status/1538271708918034433

# Tweet saved to markdown/jose_edil-1538271708918034433.md
kbravh commented 2 years ago

@gapmiss Thanks for reporting this! Let me dig in and see what I can find.

kbravh commented 2 years ago

@gapmiss The issue was occurring because of how one person being mentioned multiple times was handled incorrectly. Here is the markdown that is now produced with version 2.1.2:

---
author: "Edil Medeiros 🚒 βœοΈπŸ΄β€β˜ οΈ 🧩"
handle: "@jose_edil"
source: "https://twitter.com/jose_edil/status/1538271708918034433"
---
![jose_edil](https://pbs.twimg.com/profile_images/669315274353561600/eHheoab4_normal.jpg)
Edil Medeiros 🚒 βœοΈπŸ΄β€β˜ οΈ 🧩 ([@jose_edil](https://twitter.com/jose_edil))

[@visualizevalue](https://twitter.com/visualizevalue) [@EvansNifty](https://twitter.com/EvansNifty) [@OzolinsJanis](https://twitter.com/OzolinsJanis) [@milanicreative](https://twitter.com/milanicreative) [@design_by_kp](https://twitter.com/design_by_kp) [@victor_bigfield](https://twitter.com/victor_bigfield) [@StartupIllustr](https://twitter.com/StartupIllustr) [@tracytangtt](https://twitter.com/tracytangtt) [@AlexMaeseJ](https://twitter.com/AlexMaeseJ) [@ash_lmb](https://twitter.com/ash_lmb) [@moina_abdul](https://twitter.com/moina_abdul) @Its_Prasa [@elliottaleksndr](https://twitter.com/elliottaleksndr) [@aaraalto](https://twitter.com/aaraalto) [@tanoseihito](https://twitter.com/tanoseihito) [@jeffkortenbosch](https://twitter.com/jeffkortenbosch) [@FerraroRoberto](https://twitter.com/FerraroRoberto) [@eneskartall](https://twitter.com/eneskartall) [@SachinRamje](https://twitter.com/SachinRamje) [@AidanYeep](https://twitter.com/AidanYeep) [@jozzua](https://twitter.com/jozzua) Here they are:

[@EvansNifty](https://twitter.com/EvansNifty)
[@OzolinsJanis](https://twitter.com/OzolinsJanis)
[@milanicreative](https://twitter.com/milanicreative)
[@design_by_kp](https://twitter.com/design_by_kp)
[@victor_bigfield](https://twitter.com/victor_bigfield)
[@StartupIllustr](https://twitter.com/StartupIllustr)
[@tracytangtt](https://twitter.com/tracytangtt)
[@AlexMaeseJ](https://twitter.com/AlexMaeseJ)
[@ash_lmb](https://twitter.com/ash_lmb)
[@moina_abdul](https://twitter.com/moina_abdul)
@Its_Prasa
[@elliottaleksndr](https://twitter.com/elliottaleksndr)
[@aaraalto](https://twitter.com/aaraalto)
[@tanoseihito](https://twitter.com/tanoseihito)
[@jeffkortenbosch](https://twitter.com/jeffkortenbosch)
[@FerraroRoberto](https://twitter.com/FerraroRoberto)
[@eneskartall](https://twitter.com/eneskartall)
[@SachinRamje](https://twitter.com/SachinRamje)
[@AidanYeep](https://twitter.com/AidanYeep)
[@jozzua](https://twitter.com/jozzua)

[Tweet link](https://twitter.com/jose_edil/status/1538271708918034433)

Since this tweet is replying to all of the users, then tagging them explicitly, their names show up twice. Unfortunately, there's no immediately easy way to deduplicate this that would work in all situations. I'll think on it some more, but for now the main issue with broken links is resolved.

Thanks again for catching this!

gapmiss commented 2 years ago

@kbravh Great, thanks for the quick fix. Does this fix the Obsidian plugin also?

kbravh commented 2 years ago

@gapmiss I'll have the Obsidian plugin patched sometime soon today, and I'll comment here when I publish it πŸ™‚

kbravh commented 2 years ago

@gapmiss Version 2.10.2 of https://github.com/kbravh/obsidian-tweet-to-markdown is now live with this fix too

gapmiss commented 2 years ago

@kbravh I've updated both ttm and the Obsidian plugin. In Obsidian, the downloaded/saved profile image path is incorrect.

Here are the results. The username is prepending the image path (Twitter/media/).

Tweet URL

---
author: "Vlad Mihalcea"
handle: "@vlad_mihalcea"
source: "https://twitter.com/vlad_mihalcea/status/1549446910527918080"
date: "July 19, 2022 1:31 PM"
fetched: "August 12, 2022 8:19 PM"
likes: 21
retweets: 5
replies: 1
---
![[vlad_mihalceaTwitter/media/2255336726-z_LLR8AX_normal.jpg]]
Vlad Mihalcea ([@vlad_mihalcea](https://twitter.com/vlad_mihalcea)) - July 19, 2022 1:31 PM

How to tunnel localhost to the public Internet

[vladmihalcea.com/tunnel-localho…](https://vladmihalcea.com/tunnel-localhost-public-internet/) [pic.twitter.com/YGAOK6Jifr](https://twitter.com/vlad_mihalcea/status/1549446910527918080/photo/1)

![3_1549446907621154830](Twitter/media/3_1549446907621154830.jpg)

[Tweet link](https://twitter.com/vlad_mihalcea/status/1549446910527918080)

Here are the plugin settings:

{
  "bearerToken": "<TWITTER_BEARER_TOKEN>",
  "noteLocation": "Twitter",
  "downloadAssets": true,
  "imageEmbedStyle": "markdown",
  "assetLocation": "Twitter/media/",
  "filename": "@[[handle]]-[[id]]",
  "tweetCompleteAction": "newTab",
  "tweetLinkFetch": true,
  "embedMethod": "obsidian",
  "frontmatter": true,
  "tags": [],
  "cssclass": "",
  "freeformFrontmatter": [],
  "avatars": true,
  "avatarSize": "",
  "condensedThread": false,
  "textOnly": false,
  "includeImages": true,
  "imageSize": "",
  "includeLinks": true,
  "includeDate": true,
  "dateFormat": "LLL",
  "dateLocale": "en"
}
kbravh commented 2 years ago

@gapmiss I've moved this over to an issue on the obsidian-tweet-to-markdown repository: https://github.com/kbravh/obsidian-tweet-to-markdown/issues/33