search

brave / brave-core-crx-packager

Mozilla Public License 2.0
41 stars 35 forks source link

Add webkit-content-filter-validator step #918

Closed antonok-edm closed 4 months ago

antonok-edm commented 5 months ago

This will block publishing for any adblock component if the list cannot be converted to the syntax needed by iOS.

~Known limitation: a few of the lists are too large for lambda payloads and can't really be checked this way; we ignore them for now. In the future we can set up an endpoint that accepts compressed payloads.~ Resolved!

ShivanKaul commented 5 months ago

@mschfh can we host the validator logic on EC2 instead of lambda? It seems like we're running into lambda issues that are proving pretty expensive (in terms of engineering time) to resolve and it doesn't look like we really have a path forward: https://github.com/brave/brave-core-crx-packager/pull/918#discussion_r1621186621

mschfh commented 5 months ago

@mschfh can we host the validator logic on EC2 instead of lambda? It seems like we're running into lambda issues that are proving pretty expensive (in terms of engineering time) to resolve and it doesn't look like we really have a path forward: #918 (comment)

The Lambda was running out of memory, please let me know if the 400 responses occur again.

mschfh commented 5 months ago

we can set up an endpoint that accepts compressed payloads

done, it supports zstd now

antonok-edm commented 5 months ago

Updated to support zstd-compressed payloads. The only ignored failure now comes from Blocklists Anti-Porn due to Too many rules in JSON array. We might do something about that in the future, but I think this is great for the time being.