AdguardTeam / Scriptlets

AdGuard scriptlets library
GNU General Public License v3.0
144 stars 29 forks source link

json-prune not working with Twitter #321

Open tumatanquang opened 1 year ago

tumatanquang commented 1 year ago

Recently, Twitter has added ads to the section Trends for you: image After checking with the Chrome DevTools, I found these requests called on the Request URL: https://twitter.com/i/api/2/guide.json P/s: I did not write the query string paramters because it was quite long. And this is the position of advertising objects: image I wrote the scriptlet rule as follows to remove the index element 1: twitter.com#%#//scriptlet('json-prune', 'timeline.instructions[1].addEntries.entries[1].content.timelineModule.items[1]') I wrote the scriptlet rule as follows to remove the entryId property: twitter.com#%#//scriptlet('json-prune', 'timeline.instructions[1].addEntries.entries[1].content.timelineModule.items[1]').entryId I wrote the scriptlet rule as follows to remove the item property: twitter.com#%#//scriptlet('json-prune', 'timeline.instructions[1].addEntries.entries[1].content.timelineModule.items[1]').item But all of them are not working, the console window does not have any stack trace even though the Filtering log is still displayed.

AdamWr commented 1 year ago

Regarding first case, if I'm not wrong, something like this should works:

twitter.com#%#//scriptlet('json-prune', 'timeline.instructions.1.addEntries.entries.1.content.timelineModule.items.1')

Second and third examples are rather incorrect - https://github.com/AdguardTeam/Scriptlets/blob/master/wiki/about-scriptlets.md#json-prune

Maybe it should be something like:

twitter.com#%#//scriptlet('json-prune', 'timeline.instructions.1.addEntries.entries.1.content.timelineModule.items.1.entryId')

and

twitter.com#%#//scriptlet('json-prune', 'timeline.instructions.1.addEntries.entries.1.content.timelineModule.items.1.item')
tumatanquang commented 1 year ago

@AdamWr So, when accessing the array elements will use the same syntax as when accessing the property elements? By the way, I would like to ask the set-constant filtering rule why not work in the following case: #%#//scriptlet('set-constant', 'devtoolsDetector', 'noopFunc') Website javascript code:

(function(){
    var devtoolsDetector= // It's too long, I'll post it here: https://pastecode.io/s/1av1p7sk
    devtoolsDetector.addListener(function (isOpen, detail) {
        if(isOpen){
                        document.write("Redirecting...");
            window.location.href = "/?ref=" + encodeURIComponent(location.href) + '&n=' + detail.checkerName;
                    }
    });
    devtoolsDetector.launch();
})();
AdamWr commented 1 year ago

when accessing the array elements will use the same syntax as when accessing the property elements?

Yes.

I would like to ask the set-constant filtering rule why not work in the following case: #%#//scriptlet('set-constant', 'devtoolsDetector', 'noopFunc')

The property must be attached to window, but devtoolsDetector is inside the function, so it's in the local scope and it's not attached to window. That's the reason why it doesn't work.

tumatanquang commented 1 year ago

@AdamWr Here is a screenshot of the json parsing result of a user's tweet request: image With promoted tweets, they all have 2 things in common:

  1. Has key value entryId of the form promotedTweet-*.
  2. There exists a promotedMetadata property (I've highlighted it in yellow).

But because the placement of promoted tweets is not fixed, selection cannot be made based on property value, and not all requests to retrieve tweets by users contain promoted tweets. So I wrote a rule based on the 5th example on json-prune to avoid mistakenly filtering tweets that aren't promoted tweets: //scriptlet('json-prune', 'data.user.result.timeline_v2.timeline.instructions.1.entries.*', 'data.user.result.timeline_v2.timeline.instructions.1.entries.*.content.itemContent.promotedMetadata') But the above rule did not work properly, it will remove all elements inside the entries instead of just the elements containing the promotedMetadata property. I tried changing the rule to delete only one child property instead of the entire parent property as shown below: //scriptlet('json-prune', 'data.user.result.timeline_v2.timeline.instructions.1.entries.*.content', 'data.user.result.timeline_v2.timeline.instructions.1.entries.*.content.itemContent.promotedMetadata') And here's the result: It will remove the entire content attribute of all elements regardless of whether the element has a promotedMetadata property or not! image It looks like this rule was incorrect or the wildcard didn't work properly or something went wrong with this scriptlet!

AdamWr commented 1 year ago

It seems to be related to this - https://github.com/AdguardTeam/Scriptlets/issues/183 (second point). If I'm not wrong, currently it works like that it removes all properties (propsToRemove) if obligatoryProps is matched. Probably at the moment it's not possible to remove only this property which contains obligatoryProps. As far as I understand, such an option will be added in the future.

c0b41 commented 1 year ago

@tumatanquang did you find another solution,?