search

edrlab / thorium-reader

A cross platform desktop reading app, based on the Readium Desktop toolkit
https://www.edrlab.org/software/thorium-reader/
BSD 3-Clause "New" or "Revised" License
1.73k stars 151 forks source link

LCP-protected EPUB deleted on closing the application in windows #787

Closed ronghester closed 4 years ago

ronghester commented 4 years ago

When app is closed epub files are deleted from the local storage. This is observed in windows app.

Log from the windows machine; when app is quit [2] r2:navigator#electron/main/sessions app will quit +3m [2] r2:navigator#electron/main/sessions SESSION STORAGE DATA CLEARED - [defaul t] +2ms [2] r2:navigator#electron/main/sessions SESSION CACHE CLEARED - [default] +60m s [2] r2:navigator#electron/main/sessions Cache and StorageData cleared (default session), now webview session... +1ms [2] r2:navigator#electron/main/sessions SESSION STORAGE DATA CLEARED - [persis t:readium2pubwebview] +0ms [2] r2:navigator#electron/main/sessions SESSION CACHE CLEARED - [persist:readi um2pubwebview] +97ms [2] r2:navigator#electron/main/sessions Cache and StorageData cleared, now qui tting... +0ms

danielweck commented 4 years ago

The log messages are not related to the publication storage area (this is just the Chromium browser engine / webview "partitions" which cache HTML/CSS etc. data)

danielweck commented 4 years ago

I wonder if the source of this "files unexpectedly removed" problem could be related to this issue as well https://github.com/readium/readium-desktop/issues/325

danielweck commented 4 years ago

So, could you please explain the bug in greater detail? Does this mean that when you re-open the app, the EPUB files are still in the library / bookshelf view, but when you click on them they do not load?

ronghester commented 4 years ago

we have observed that when a windows app is closed the book contents of the book which are in catalog are getting deleted. Specifically if we are in reader window and close the app then that books content (epub) is surely getting deleted. When we re-open the app the book is displayed in the library/bookshelf and when we click on it nothing happens there is reader error in console. On further debugging this i found that the epub file gets deleted/removed from the user localstorage directory when we close the app. This is only happening in windows app and not in Mac.

danielweck commented 4 years ago

when we click on it the epub is re-downloaded

What do you mean? This is not OPDS, right? (just a regular import from the local filesystem)

ronghester commented 4 years ago

when we click on it the epub is re-downloaded

What do you mean? This is not OPDS, right? (just a regular import from the local filesystem)

Sorry i have corrected my response. It does not re-download nothing happens when we click on that book. There is following error shown in the console.

[2] r2:streamer#http/server Error: ENOENT: no such file or directory, lstat 'C :\Users\15728\AppData\Roaming\thorium\publications\30a38547-ac86-4286-974f-ace0b 4bd2995\book.epub' [2] r2:streamer#http/server at fs.lstatSync (fs.js:936:3) [2] r2:streamer#http/server at Object.fs.lstatSync (ELECTRON_ASAR.js:259:1 6) [2] r2:streamer#http/server at Object. (C:\Users\15728\r2app\re adium-desktop-develop\node_modules\r2-utils-js\dist\es6-es2015\src_utils\zip\zi pFactory.js:21:26) [2] r2:streamer#http/server at Generator.next () [2] r2:streamer#http/server at C:\Users\15728\r2app\readium-desktop-develo p\node_modules\tslib\tslib.js:107:75 [2] r2:streamer#http/server at new Promise () [2] r2:streamer#http/server at Object.awaiter (C:\Users\15728\r2app\read ium-desktop-develop\node_modules\tslib\tslib.js:103:16) [2] r2:streamer#http/server at Object.zipLoadPromise (C:\Users\15728\r2app \readium-desktop-develop\node_modules\r2-utils-js\dist\es6-es2015\src_utils\zip \zipFactory.js:12:20) [2] r2:streamer#http/server at Object. (C:\Users\15728\r2app\re adium-desktop-develop\node_modules\r2-shared-js\dist\es6-es2015\src\parser\epub. js:133:38) [2] r2:streamer#http/server at Generator.next () [2] r2:streamer#http/server at C:\Users\15728\r2app\readium-desktop-develo p\node_modules\tslib\tslib.js:107:75 [2] r2:streamer#http/server at new Promise () [2] r2:streamer#http/server at Object.awaiter (C:\Users\15728\r2app\read ium-desktop-develop\node_modules\tslib\tslib.js:103:16) [2] r2:streamer#http/server at Object.EpubParsePromise (C:\Users\15728\r2a pp\readium-desktop-develop\node_modules\r2-shared-js\dist\es6-es2015\src\parser\ epub.js:120:20) [2] r2:streamer#http/server at Object. (C:\Users\15728\r2app\re adium-desktop-develop\node_modules\r2-shared-js\dist\es6-es2015\src\parser\publi cation-parser.js:8:57) [2] r2:streamer#http/server at Generator.next () +633ms [2] ### action type STREAMER_PUBLICATION_OPEN_ERROR [2] ### action type READER_OPEN_ERROR

danielweck commented 4 years ago

Indeed, this error seems very similar to https://github.com/readium/readium-desktop/issues/325 :( Error: ENOENT: no such file or directory, lstat C:\Users\ME\AppData\Roaming\thorium\publications\30a38547-ac86-4286-974f-ace0b 4bd2995\book.epub

danielweck commented 4 years ago

Are you experiencing the bug with LCP-protected publication only, or with non-LCP EPUBs as well? If the former, then this is a duplicate issue with https://github.com/readium/readium-desktop/issues/325 (in which case we can close this present issue) If the latter, then we have a new problem (in which case we can keep this present issue open)

ronghester commented 4 years ago

Yes I have experience this problem with LCP protected Epubs. i think #325 is totally different issue its happening when the books is opened for reading and it is reported on mac and windows as well. I have never seen 325 and don't think it is replicable. This issue however has a clear replicable steps.

  1. Import book in windows app.
  2. Open book from reader (You should see the book content in reader).
  3. Close the reader (or go back to the home screen).
  4. Close the app.
  5. Navigate to the localstorage directory where the book was stored. You will see the book epub is removed/deleted by that app.
  6. Open the app again.
  7. Open the book .. [Nothing will happen]. Error in console that book is not found. Which is obvious because its deleted by the app itself when it was quit last time.
danielweck commented 4 years ago

The LCP code is currently incomplete (in fact, some of it has been disabled) so I am not surprised that there are some differences with the previous issue. Let's keep them both open, but let's be more explicit about LCP, as I believe this bug does not occur at all with non-LCP publications.

danielweck commented 4 years ago

In your tests, are the LCP-protected EPUBs downloaded from the LCP licenses (e.g. book.lcpl file), or are you able to replicate the bug when importing LCP-protected EPUBs directly (i.e. not via standalone LCP license files)?

ronghester commented 4 years ago

Yes i can confirm that this issue occurs only on LCP protected epubs. While testing the windows app i am supplying the book.lcpl and then the book epub is getting downloaded to the local file directory. Interestingly only the epub file is removed and license file is not removed/deleted.

ronghester commented 4 years ago

Further Update : After some more testing; it is found out that the book.epub is only deleted when that book is open in the reader. If the book is not opened after importing it in that case the book.epub is not removed. So it is important to note the removal of the file takes place only if it is decrypted and opened in the reader.

Code Debugging After looking at the reactjs code of reader/streamer it is clear that no file operation is performed in reactjs. The server is started and content is served from the streamer. The decryption and resource server is started in c++ code & lcp.node. So may be the problem is in lcp.node file.

Conclusion Since LCP.node deals with the file decryption etc.. it might be the case that it opened the file and failed to release the file pointer. However Since lcp.node fie can't be debugged it would be difficult to conclude this.

danielweck commented 4 years ago

The LCP native NodeJS plugin does not access the filesystem, it only manipulates in-memory buffers of binary data.

danielweck commented 4 years ago

This is fixed by this WIP PR: https://github.com/readium/readium-desktop/pull/795

danielweck commented 4 years ago

I think this Windows filesystem bug persists: https://github.com/readium/readium-desktop/issues/884