search

GladistonXD / perlego-download

Google chrome extension
39 stars 10 forks source link

Memory problems #4

Closed dresatt closed 10 months ago

dresatt commented 10 months ago

My browser crashes because of memory usage (the book is large), but It's important for my course When I try to refresh the browser, the -- / -- page found popup doesn't show up and it runs out of memory soon after. I have around 500 pages in html but another 500 pages to go, can you please implement an option to make it start at a certain page? Thank you

GladistonXD commented 10 months ago

My browser crashes because of memory usage (the book is large), but It's important for my course When I try to refresh the browser, the -- / -- page found popup doesn't show up and it runs out of memory soon after. I have around 500 pages in html but another 500 pages to go, can you please implement an option to make it start at a certain page? Thank you

This option was implemented in the last update, it continues automatically, just restart the page and click again to continue, it's all saved in the browser cache, read the description for low memory tips.

H9RuH2d commented 10 months ago

Hey, I had the same issue. I think the crash is related to the extension, not the Perlego website, so while the update that allows us to continue where we left off is nice, I don't think it will help with the crash.

What I did to circumvent the crash was I saved the first 300 pages of my book as HTML, then I edited this value to 300, so that the next time it would start from page 300.

So I ended up with 2 HTML files -> 2 PDF files, then I combined the 2 PDF files into one. If you're on macOS you can use this to combine 2 PDFs.

GladistonXD commented 10 months ago

Hey, I had the same issue. I think the crash is related to the extension, not the Perlego website, so while the update that allows us to continue where we left off is nice, I don't think it will help with the crash.

What I did to circumvent the crash was I saved the first 300 pages of my book as HTML, then I edited this value to 300, so that the next time it would start from page 300.

So I ended up with 2 HTML files -> 2 PDF files, then I combined the 2 PDF files into one. If you're on macOS you can use this to combine 2 PDFs.

With a very large flow of pages, the browser will crash depending on the machine you are using, my tip was to edit the .html file manually and divide these pages directly from the final file with a text editor and divide the file into parts, or open the file in Firefox, which works better with data flow and sometimes does not require division.

H9RuH2d commented 10 months ago

Just to be clear, are you talking about a crash when opening the generated HTML, or a crash while actually generating the HTML (while using the extension on Perlego)?

I believe both me and OP are talking about a crash while using the extension on Perlego.

GladistonXD commented 10 months ago

Just to be clear, are you talking about a crash when opening the generated HTML, or a crash while actually generating the HTML (while using the extension on Perlego)?

I believe both me and OP are talking about a crash while using the extension on Perlego.

Is the failure related to a crash when executing the code? it doesn't make sense since the saved file does not directly influence the page it adds data externally, crashes can occur when perlego tries to find the page /0 when opening the Table of contents this occurs with or without the extension, in theory if you stop on the page 300 update the page at /300 the code is to continue without crashing, since the number of pages loaded is kind of reset in the scroll bar, but some bug may occur that I didn't see when testing.

H9RuH2d commented 10 months ago

I don't think it's crashing because of the /0 thing. Here's the steps to reproduce.

  1. Open book in Perlego to page 1.
  2. Choose PDF option in extension.
  3. Progress bar shows 1/700.
  4. Keep scrolling until 200/700.
  5. Page crashes (browser shows generic error message for the page).
  6. Reload the page, setting URL to /200.
  7. Choose PDF option in extension.
  8. Extension remembers where it left off, progress bar shows 200/700.
  9. Keep scrolling until 220/700.
  10. Page crashes (browser shows generic error message for the page).
  11. Reload the page, setting URL to /220.
  12. Extension remembers where it left off, progress bar shows 220/700.
  13. Within a few pages, crashes again. At this point, every time I repeat the steps it crashes very quickly.

If I quit/save the HTML early, the page no longer crashes. To me this looks like it's related to the extension.

H9RuH2d commented 10 months ago

Where is the extension keeping the HTML before writing it out to a file at the end? Is it in memory? In browser cache? I don't really know much about browsers/web dev so can't quickly skim the code to answer this for myself.

After splitting the process in 2 as I mentioned in an above comment, I ended up with two 90MB files, so the total size of HTML would be 180MB if able to get it in one go. Have you tested any books with similar file size?

GladistonXD commented 10 months ago

It saves in an IndexDB database. In theory, it is possible to allocate up to 150 GB with this method, unless you don't have space on the disk where Chrome was installed, but once you have time, I'll add an option to start from wherever you want.

GladistonXD commented 10 months ago

I implemented a page flow division within the database, in theory this prevents memory leaks, test it and see if it solves the problem. @H9RuH2d @dresatt

H9RuH2d commented 10 months ago

It seems to have fixed it, thanks!