jpd236 / CrosswordScraper

Browser extension which downloads crosswords from crossword applets for offline solving.
Apache License 2.0
29 stars 1 forks source link

Migrate to manifest v3 for Firefox; delete v2 support #4

Open jpd236 opened 3 years ago

jpd236 commented 3 years ago

I think the main question here will be how dangerous it is perceived to declare broad host permissions with v3. Optional host permissions seem to be impossible with v3, which would make this extension look much more dangerous. We need this in order to be able to fetch .puz or other puzzle links from arbitrary pages.

May be other minor differences, but will need to test.

jpd236 commented 3 years ago

Chrome announced they plan to fully disable MV2 extensions in January 2023 per this timeline: https://developer.chrome.com/docs/extensions/mv3/mv2-sunset/

The optional host permission piece still seems important here - last update I could find at https://groups.google.com/a/chromium.org/g/chromium-extensions/c/u5tRDEn8yjU/m/PZn4o4ZlAgAJ suggests this is still in progress, but delayed. If push comes to shove, we could probably just migrate to a fixed list of known sites (though I wonder what the experience is when you add a new host permission in an incremental update?)

Another concern is that executeScript no longer supports arbitrary code. Some scrapers require injecting a script to access variables in the host page. That said, it does support either static .js or a function reference, which I think works here since we're accessing known JS variables rather than dynamic ones, so we should be able to make something work.

Lastly, Firefox doesn't yet support Manifest v3, so will either need to wait for that or figure out a way to target both.

jpd236 commented 2 years ago

Optional host permissions have been added to Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1265064. (Still needs to actually be rolled out / accounted for in the docs, but this seems like a good sign).

jpd236 commented 2 years ago

Got something working for a few different sources; seems like the approach should generalize: https://github.com/jpd236/CrosswordScraper/commit/96971e52637b6ad0174bac2608d6ae0993c5a2eb

This requires defining a .js file for each snippet of JS we want to run; it's functional, but ideally we'd be able to use the alternate approach where you pass in a function which would let us write the injected code in Kotlin. Would need to find some way to make the Kotlin function addressable from the JS.

Will probably end up needing to fork off separate Chrome/Firefox source sets unless Firefox gets their manifest v3 support in order relatively soon.

jpd236 commented 2 years ago

We can now target both v2 and v3, and both are being built continuously. The v3 extension appears to be fully functional in Chrome 102 (as is required for optional_host_permissions). I haven't tested in Firefox since v3 support is further away there, but I don't see any reason for it not to work once we migrate from chrome.scripting to browser.scripting (hopefully via polyfill), and once Firefox supports optional_host_permissions (https://bugzilla.mozilla.org/show_bug.cgi?id=1766026) and any other missing pieces in its implementation.

While we could push this to the Chrome App Store immediately, I intend to hold off until closer to the v2 deprecation, or if there becomes a major incentive to upgrade (like a security warning shown to users or significantly slower reviews on v2), given that Chrome 102 is a relatively recent version and we should give the update time to propagate and for v3 support to continue to stabilize.

jpd236 commented 2 years ago

Now that Chrome 102 has been out for a few months, I'll go ahead and start using the v3 extension in the Chrome Web Store.

Remaining work here is to migrate Firefox to v3 if/when they reach a point of fully supporting what we need, and then to clean up the legacy v2 codepaths.

jpd236 commented 1 year ago

Firefox 109 supports v3 extensions, but not ExecutionWorld.MAIN (https://bugzilla.mozilla.org/show_bug.cgi?id=1736575) which is needed for most scrapers to work (anything reading from Javascript context, though DOM access still works, IIUC). There might be workarounds (using the old v2 method) but since the main benefit from my perspective of v3 for Firefox is unifying the implementations with Chrome and Firefox, I don't see much purpose.

See also https://bugzilla.mozilla.org/show_bug.cgi?id=1766026 re: optional host permissions (which look like they might work, just under optional_permissions for now). And I believe we could get rid of webextension-polyfill as part of this and just use the chrome.* API namespace everywhere, but would need to test carefully.

jpd236 commented 6 months ago

Note that Android support for Manifest v3 is behind desktop; see https://extensionworkshop.com/documentation/develop/developing-extensions-for-firefox-for-android/#mv3-compatibility for some known issues.