Extension only sees tweets rendered client-side, doesn't block users rendered server-side

[camjackson]

Browser: Firefox 114.0.2 Extension: v0.3.3

I'm looking at this tweet, and it seems no users are being added to the queue or being blocked as I scroll. The extension says:

blocked 0 users so far! (0 queued: view)

Console logs when I load that tweet:

[Blue Blocker] queue consumer started
[Blue Blocker] parsed instructions path: Array [ {…}, {…} ]
[Blue Blocker] parsed instructions path: Array [ {…}, {…} ]
[Blue Blocker] queue consumer stopped

I tried to paste the full JSON contents of those two arrays, but they're too long to fit here. However, I did note that every user (in both arrays) has "is_blue_verified": false, despite there being plenty of blue checks in the replies. That seems suspicious?

Service worker logs (couple of errors at the bottom but I don't think they're relevant?):

[Blue Blocker] upgrading history db: 
IDBOpenDBRequest { onblocked: onblocked(), onupgradeneeded: onupgradeneeded(), result: IDBDatabase, error: null, source: null, transaction: null, readyState: "done", onsuccess: onsuccess(), onerror: onblocked() }
[Blue Blocker] created history database.
[Blue Blocker] successfully connected to history db
[Blue Blocker] opening legacy verified user database: 
IDBOpenDBRequest { onblocked: onblocked(), onupgradeneeded: onupgradeneeded(), result: IDBDatabase, error: null, source: null, transaction: null, readyState: "done", onsuccess: async onsuccess(), onerror: onblocked() }
[Blue Blocker] legacy db onupgradeneeded: 
IDBOpenDBRequest { onblocked: onblocked(), onupgradeneeded: onupgradeneeded(), result: IDBDatabase, error: null, source: null, transaction: null, readyState: "done", onsuccess: async onsuccess(), onerror: onblocked() }
[Blue Blocker] created legacy database.
[Blue Blocker] successfully connected to legacy db
[Blue Blocker] checking verified user database.
[Blue Blocker] cleared existing legacyDb store.
[Blue Blocker] downloading legacy verified users database, this may take a few minutes.
[Blue Blocker] response csv good!
[Blue Blocker] stored 1,000 legacy verified users 407
[Blue Blocker] committed 407520 users to legacy verified legacyDb: 
IDBTransaction { mode: "readwrite", db: IDBDatabase, error: null, onabort: null, oncomplete: null, onerror: null, objectStoreNames: DOMStringList(1) }
[Blue Blocker] loaded 407,520 legacy verified users!
[Blue Blocker] soupcan response for @elonmusk: Error: Could not establish connection. Receiving end does not exist.
    <anonymous> moz-extension://4f448140-acc7-48d5-b3dc-ad1236d73112/assets/chunk-6e98eb87.js:3
[Blue Blocker] soupcan response for @elonmusk: Error: Could not establish connection. Receiving end does not exist.
    <anonymous> moz-extension://4f448140-acc7-48d5-b3dc-ad1236d73112/assets/chunk-6e98eb87.js:3
[Blue Blocker] soupcan response for @elonmusk: Error: Could not establish connection. Receiving end does not exist.
    <anonymous> moz-extension://4f448140-acc7-48d5-b3dc-ad1236d73112/assets/chunk-6e98eb87.js:3
[Blue Blocker] soupcan response for @elonmusk: Error: Could not establish connection. Receiving end does not exist.
    <anonymous> moz-extension://4f448140-acc7-48d5-b3dc-ad1236d73112/assets/chunk-6e98eb87.js:3

[kheina]

hey thanks for all this, this is really good info. yeah it looks like your worker is fine, but you're correct that that is_blue_verified is what bb is looking for. I'll have a look at this in depth tomorrow and try to see what's up. I might have extra questions or info for you then

[kheina]

I will mention before I go, that you can download logs from your console and paste them as files here. just double check there's no sensitive info in them first

[camjackson]

Ok a bit more info. It looks like the issue is that the extension only sees tweets that are loaded after the initial page load. Here are my observations:

1. Initial page load. Log says only [Blue Blocker] queue consumer started
2. Start scrolling. Log says [Blue Blocker] parsed instructions path:. The two instruction types are:
   1. TimelineAddEntries
   2. TimelineTerminateTimeline
3. All the blue check users are in the first 5 or so replies, bc twitter sorts blue users to the top
4. Digging into the JSON, the first user loaded as part of the AddEntries instruction is the 11th reply.

So basically the extension doesn't see any users that are already on the page when it loads, which all of the blue users are. The extension only sees the users "below the fold" so to speak, who are all non-blue.

[camjackson]

Ok further test results confirm that the problem is due to initial page load vs. tweets fetched after page load.

If I load the tweet by opening up a new tab and navigating directly to the tweet permalink, the behaviour is as described above. If however I am already on twitter and load that tweet via a client-side transition (e.g. click on a link to the tweet from a quote-tweet, from search results, or from the user's profile), then everything works fine.

Maybe the extension was always intended to work this way. I realise it's much easier to sniff XHR payloads than to crawl the DOM. Would be cool if it could do both though!

Gonna rename this issue to be more specific about the actual problem. It's probably a feature request rather than a bug.

[kheina]

I looked into this a little bit earlier and yeah that's the issue. blue blocker does work by xhr hijacking and I think it's just a firefox order of operations thing, since on chrome this isn't an issue, it just means that twitter's frontend was able to send the xhr request before blueblocker was able to bind to the xhr prototype. I have known about this issue for a while but haven't been able to find a fix for it yet.

[camjackson]

Ahhhh ok got it. Do you think it would be worth adding like a "known issues" section to the wiki? I could open a PR if you're interested?

[kheina]

it's probably worth making a known issues section yeah. if you know of a fix for this issue on firefox then absolutely make a pr! xhr is modified in the inject script, which is run by the startup content script

[kheina]

occurs to me you meant pr to the wiki, yeah go for it lol I'd just make a new page for it