search

danny0838 / webscrapbook

A browser extension that captures web pages to local device or backend server for future retrieval, organization, annotation, and edit. This project inherits from legacy Firefox add-on ScrapBook X.
Mozilla Public License 2.0
850 stars 118 forks source link

Fatal Error (invalid URL) during rebuilding #374

Closed FalxGod closed 4 months ago

FalxGod commented 4 months ago

I'm getting the following error when using webscrapbook and I have no idea how to fix it.

Rebuilding links...
Fatal error: URL constructor: https://*************/d0I0Mb1bub/Interactive__setup_v1.1_var is not a valid URL.

The URL doesn't even match my regex for included websites so I dont know why the rebuilding would fail.

The link is embedded like this: <a href="https://**********.com/" target="_blank" class="link link--external has-favicon" rel="nofollow noopener" style="background-image: url(&quot;https://external-content.duckduckgo.com/ip3/.ico&quot;);">https://**********.com/</a>

I have no idea how to prevent webscrapbook from crashing if it encounters an invalid URL. Help would be appreciated :)

Edit: I already added the following regex to "Excluded URLs": /^https:\/\/\*.*/

danny0838 commented 4 months ago

We cannot reproduce the issue according to the provided info. Please provide more details, including the version of the browser and WSB, the capture options (which can be copied from capture as > advanced), and the URL of the source page (or, alternatively, the minimal HTML file(s) that can trigger the same issue when hosted locally).

FalxGod commented 4 months ago

I'll gladly provide more info: Tested on the following browsers: -Firefox 123.0 (x64) -Firefox 123.0.1 (x64)

WSB backend server version: 2.3.0 WSB browser extension version:

The Fatal Error occurred when using the backend server and when not using it.

Sadly the source page is behind a login so here's the HTML snipped that I hosted on an apache and managed to reproduce the error: https://www.mediafire.com/file/msoikskb7ic5mvt/test.html/file

And here are my capture options: https://www.mediafire.com/file/cns2au2xbvh1yn1/wsb_options.txt/file

I also tried using Chrome once, and the fatal error doesn't occur there. I need more testing to ensure this is a FIrefox only error first though. It might take a while because I'm at work at the moment

danny0838 commented 4 months ago

We still cannot reproduce the issue using the file you provided.

After some further investigation it seems that any invalid URL can trigger the issue, and version 2.8.2 should have fixed the issue if our guess is correct. Please confirm it. If the issue still persists, please provide the file without masking content (which is likely the source of the error) or provide an non-masked file that can really trigger the issue for further investigation.

FalxGod commented 4 months ago

After extensive testing yesterday and today, I can confirm that version 2.8.2 fixed the fatal error. Thank you danny you absolute GIGACHAD!