[WIP] Add support for Markdown in tweets

[jorgegonzalez]

Fixes #39 Fixes #41 Fixes #108 Fixes #110 Closes #109 Fixes #123

@filipekiss I'm sorry that this duplicates all of the work you've been doing in #109! The current solution we had for the inline code and the highlighted code blocks was pretty messy. This PR should finally fix the issues with links in the styled code. It also adds support for all other markdown.

It should also be noted that remark-html, remark-parse, and unified are adding about 100KB to the bundle, so this may not be worth it, though the packaged chrome extension will only go from 50KB to 92KB, which isn't too crazy.

Todo:

• [x] Fix highlighted code padding
• [x] Fix header padding
• [x] Style blockquotes
• [x] Style lists
• [x] Possibly tweak strong and em elements
• [x] Possibly strip http:// from links in inline code
• [ ] Fix styles in tweet modals
  • https://twitter.com/rauschma/status/998556286538133505
• [ ] Remove link cards if they originate from code blocks Use smart quotes https://twitter.com/zats/status/996730907192102912

Issues:

• [x] Some links in the stream are getting squashed
  • Ignoring as I couldn't reproduce
  • Was related to the issue below, now fixed
• [x] Space before tags in non-processed tweets
  • Fixed by ignoring link tags in non-markdown tweets
• [x] Code blocks are not styled when language isn't specified
  • https://twitter.com/sindresorhus/status/994307954030526464

https://twitter.com/mikeal/status/1011406440999763968

Won't work on this PR anymore until the bundle size issue is discussed.

[jorgegonzalez]

[jorgegonzalez]

[filipekiss]

@jorgegonzalez No problem at all! This is a great solution and also adds Markdown support, which I'm 100% ok with!

[filipekiss]

About the bundle size: I think it's negligible. The value added is enormous and the extension itself isn't big (although it almost doubles size).

The extension will be installed and then done. There's no need to worry about size so much because there are no network requests or anything like that after the extension is installed/updated, so I'm perfectly fine with it.

[jorgegonzalez]

We are currently formatting the smart quotes style (and the little guys hands), anyone know why this is? twitter.com/zats/status/996730907192102912

[filipekiss]

We're not actively doing this. Probably when we parse the tweet to add markdown support (or something), using Element.textContent get's rid of the smart quotes.

[jorgegonzalez]

The master branch does it too, and when inspecting the <p> element, the smart quotes are still there.

Not a major issue though, I think.

[filipekiss]

Yeah, the master branch may be doing via the old code-highlight.js file. I'll see if I can find what commit introduced theses changes later doing a bisect.

[sindresorhus]

This is amazing! I'm ok with the size increase.

[jorgegonzalez]

Awesome, I'll work out the kinks soon.

[jorgegonzalez]

Possible issue https://twitter.com/rauschma/status/998556286538133505

'a|b|c'.split`|`

[filipekiss]

Not sure if it's actually an issue. That's not standard, it's just abusing the syntax/language to show how stuff works. I wouldn't mind much.

[jorgegonzalez]

Yeah I don't mind either, just trying to document encounters like this.

[jorgegonzalez]

@'s are not being tokenized properly.

[jorgegonzalez]

this is a test

[jorgegonzalez]

https://twitter.com/DiscreteQuantum/status/1005474353151127552

[jorgegonzalez]

I think this PR is close to finished. I need to tweak a few more things, but I'd appreciate it if anyone wants to start testing the branch! Also any feedback.

[jorgegonzalez]

Multiline code blocks are getting smushed. CSS is hard.

[jorgegonzalez]

https://twitter.com/styfle/status/1008762725282000896

[sindresorhus]

Great progress, some quick notes:

• In dark mode, bold text has the wrong color.
• Missing linebreaks content.

[jorgegonzalez]

Emojis are being removed https://twitter.com/MartijnSaly/status/1011223724496957441

[jorgegonzalez]

Sorry this is taking so long. In the time since I opened this PR I graduated university, went on holiday, moved to a new city, and started a new job. Will try to work on it more this weekend!

[jorgegonzalez]

So emojis inside of inline code blocks (like this 🔥) are being deleted. I can't really figure out why.

I feel like that's not a common use case. Should we fix that issue in a different PR?

[filipekiss]

@jorgegonzalez I'm okay with it being deleted for the time being and we work on that later.

[notlmn]

Emojis are converted to <img> tags by Twitter, so you're almost screwed over there.

When you call element.textContent, the result would be

Okay okay, then how about calling `` an error and `` an event. Hey javascript allows it!
                                   ^-- emoji gone from here

You'd need to specially handle DOM nodes of the tweet, and serialize them!

[jorgegonzalez]

Might be able to write a tokenizer for this.

[jorgegonzalez]

https://twitter.com/swyx/status/1023762646758240256

[jorgegonzalez]

Some CSS tweaks (not my strength) need to be made, and there are still some issues with links, and the escape character.

[superhawk610]

I have another failing example, it may be fixed by the existing changes in this PR, not sure.

Original text:

Reading up on Object.is vs === - is there any world where I would care to differentiate between +0 and -0? What about NaN and Number.NaN? Why?

@jorgegonzalez @filipekiss what's the current status on this PR? I may be able to help wrap it up, just want to confirm that no one else is actively working on it.

[jorgegonzalez]

I had been actively working on it but stopped due to various issues I kept running into, and lack of time as I started a full time job. The branch is open and I’ll merge anything you want to contribute.

[superhawk610]

Alright, I'll track my progress here:

• [ ] retweet icon is missing in Timeline and Tweet modal
• [ ] quotes only take up a few pixels in height
• [ ] extension errors on load in Chrome

  Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self' blob: filesystem: chrome-extension-resource:".

[jorgegonzalez]

@superhawk610 any updates?

I've had less and less time as the year has gone on to work on this.

[superhawk610]

After an initial bit of work, working on this keeps losing out to interview prepping 😬 I would still like to finish this, but if anyone else is able to contribute sooner than me then that would also be great.

I'll try to put some time towards it this weekend.

[jorgegonzalez]

working on this keeps losing out to interview prepping

I'm right there with ya, bud.

[sindresorhus]

I'm closing this for now as this probably broke with the new Twitter redesign and it hasn't had any activity for some time.