search

theAlinP / twitter-link-deobfuscator

A Firefox add-on that restores the original destination of the links (from tweets) that have been shortened by the Twitter servers. It only runs while browsing Twitter's website (twitter.com).
https://addons.mozilla.org/en-US/firefox/addon/twitter-link-deobfuscator/
MIT License
36 stars 5 forks source link

Deobfuscation does not work for Twitter cards #1

Closed muellermartin closed 5 years ago

muellermartin commented 5 years ago

The deobfuscation does not work for Twitter cards which are embedded into tweets. For example this tweet shows the problem: https://twitter.com/mozilla/status/1111330976700006403

This is because Twitter Cards use iframes and the extension currently is not able to descend into the iframe.

It would be really nice, if the extension could also deobfuscate links in Twitter Cards.

theAlinP commented 5 years ago

The Twitter servers replace the <a> elements' href attribute values with shortened URLs but, fortunately, they add the original URLs in data-expanded-url attributes and the title attributes. If Twitter Link Deobfuscator finds any hyperlinks with data-expanded-url attributes, it replaces each of their href's attribute value with the data-expanded-url attribute's value then deletes the second attribute.

I did not allow TLD to descend into iframes because I came to the conclusion that there are no hyperlinks with data-expanded-url attributes there, only hyperlinks that point to, and resources from hosts owned by Twitter (t.co, pbs.twimg.com, ton.twimg.com).

However, looking closer, I just noticed that the hyperlinks from the Twitter cards can be cleaned, also. The original URLs are there, not in the iframes but in the parent document, in hidden sibling hyperlinks above the iframes.

That means that I could probably make it work with Twitter cards, it's just that I would need some time.

muellermartin commented 5 years ago

Before creating this issue I took a glimpse at the source code and saw that the plugin simply takes the URL from the mentioned attribute.

Nice to hear, that descending into the iframe is not neccessary. I've never developed Firefox extensions, but if I can help with a PR or testing, feel free to ask for help :)

theAlinP commented 5 years ago

Hey, Martin

I haven't forgotten about this issue and I thought about it a little bit every day.

So far I managed to make TLD detect the Twitter Cards and to deobfuscate the first 2. Why 2?

The tweets are created in batches. A couple of dozens are created the first time the page is loaded then others are downloaded as the user scrolls to the bottom of the page. Even though the Twitter Cards are created at the same time with the other tweets, the iframes inside them are not. It turns out that the iframes are created dynamically as the page is scrolled down and their parent divs come into view. That's why TLD can only see a couple of iframes and clean the links inside them when the page is loaded.

I will need to update TLD to clean the links from the iframes created as the page is scrolled down. I will make a new commit and publish a new version on TLD's AMO (addons.mozilla.org) page in a few days.

theAlinP commented 5 years ago

I finally added support for Twitter Cards.

What I thought last time would take me a couple of days turned out to be a dead end.

At first I tried to modify the iframes from a content script injected into the top level document. Even though I could modify the first iframes created on page load, I could not find a way to modify the ones created as the page is scrolled down. The same function that modified the iframes created on page load could not do that if it was invoked by a Mutation Observer, when new iframes were created. I found no explanation as to why that would happen.

Many many hours later I eventually abandoned that approach and took a different path. I allowed the content script to run in iframes, too, and from there it contacts the background script, exchanging messages several times to get the original URLs and finally modify the iframes from inside.

Please update the add-on and tell me if it works properly in your browser so I can close this issue.

muellermartin commented 5 years ago

I've updated the add-on to version 0.3.0 and it works like a charm :)

Thanks for your efforts and the detailed description of your solution!