Closed astarq closed 2 years ago
astarq
commented
4 years ago When I saw you mentioned DNS-based content blockers, I thought that might be the problem as the one I'm currently using blocks a few Google URLs, but disabling doesn't seem to make a difference 🙁
johnozbay
commented
4 years ago Hi there! 👋🏻
Thank you so much for filing this with such great detail! And many million thanks for all your help with the PWA bugs on the Mozilla Fenix branch as well. I ran into one of your reports trying to file the same bug last night. Firefox PWA fans owe you a lot 🙏🏻
I'll take a look and get this fixed right away. Sounds incredibly frustrating indeed.
A little background here – epub ebooks are actually simply well structured html files. This means they're great for viewing online. But as a side effect, the epub format itself comes with a big potential security risk if you're viewing it online/in a browser. Because ebooks could theoretically contain malicious code in them, and if we were to open them without proper sandboxing, these could potentially compromise your account security in some way.
Due to this, we sandbox them and load them in iframes, then exchange page flip events.
I have two guesses here.
1) The ebook library isn't passing the page-flip/bookmark events correctly through the frame for some reason, so it loses track of where you are, then Cryptee can't can't save your progress/bookmarks.
2) Before saving a page progress / bookmark etc, we run some basic checks to make sure your local docs/folders catalog is still intact. i.e. the book is open on your phone, you're reading, and simultaneously, on your desktop, in Cryptee Docs you delete the folder containing the book. Theoretically, the book 'could' remain accessible on your phone for another second or two. (We use sockets to notify all devices for changes like these to prevent things from breaking) – But if for some reason, while you're reading the book, you can manage to magically delete the folder the book is in, and try to save a page-flip, it could break a lot of things in your account, since the app wouldn't know where to save the page-progress/bookmarks etc (as the source path is now deleted)
Here's where the check is :
that fid on line 11091 is short for folder id.
rarely, due to a combination of network issues and on-device encryption, your local catalog could get out of sync / out of date. Cryptee does its best to correct 99.9% of these, but real-life network issues are a real pain to solve when everything is encrypted on the device, and servers can't help facilitate conflict resolution.
And since your data is on-device encrypted, we can't remotely help you fix these either, so instead we've made a self-help/first-aid-kit button. Head over to docs preferences in your account settings, and at the bottom you'll see "Local Cache" – Clear :
Try pressing this, and let's see if this solves the problem. If it does solve the problem, then chances are some strange network issue messed up with your device's local encrypted cache, and that's what's been causing these problems. I'll build more failsafes to ensure ebooks can do online checks and keep working.
If not – in the meantime I'll run some real-life hands on tests with the Galaxy we have here, and get back to you on this as soon as I find out more.
Many million thanks for your help once again ✌🏻
Best, J
astarq
commented
4 years ago Hi!
You're most welcome, and thanks for the reply! I have little - very little - coding experience, so some of it went straight over my head :) And it's always nice to be recognized in ones work 🤗
Sadly, I'd already tried using Cryptee's cache clear button, but it didn't seem to do the trick. Also, my changes weren't saved in other browsers - or even after a factory reset (don't ask!)
If I can help out on this in any noob-friendly way, I'd be happy to. Apologies in advance if I take a while to reply, I'll get back to you eventually!
johnozbay
commented
4 years ago Hey there! First off – no worries about coding experience at all, and apologies for the code-heavy explanation.
I'm infinitely thankful that you've found & reported many critical bugs, both here and at mozilla's repos. – despite your little coding experience – Massive kudos and thank you! The world needs more amazing people like you!
As for the bug! I asked everyone on the team to go read a book for the week. (Turns out none of us had enough time to properly sit down and read books, so now we know what the real problem was hahaha) And I think we've figured out what's going on.
I'm not entirely sure if we can fix this yet (or how we can fix this in the first place) – but at least we understand the problem better. In short, the bug has to do with different device / screen sizes.
So to explain this better, it's best to think of an ebook like a one-gigantic roll of star wars text :
instead of something that has pages like a physical book. Let's say you're reading The Metamorphosis by Franz Kafka – which is 21,810 words in total.
If you're using a small-screen device, with larger fonts to read easily etc. let's say you can fit about 100 words on your screen, which means on mobile The Metamorphosis by Franz Kafka will be ~218.1 pages (rolled into 219 pages).
Whereas on a desktop, let's say you can fit 200 words, with a smaller font. Meaning that you'll have 109.05 pages (rolled into 110 pages).
And this assumes there's a perfectly balanced distribution. When the font size / margin size / screen size changes, having even one word in a previous line could result in a one-less line per page, and with a book that's 219 'pages', even if you get one-less line per page, and there are 50 lines on each page, you'll get something like 4 pages of difference.
Here's why this matters. When we bookmark your position in a book, or save your progress, we have to save it based on lines, and save something like : "here's the first line visible on the page" and reference this line in the book to save your progress (instead of page numbers etc, since that's all virtual and subject to change with ebooks)
From what it looks like, if you switch between two different screen sized devices (this could even be two different browsers on same device with more/less screen real estate due to larger address bar etc.) the progress calculation eventually gets messed up, depending on the book, whether if all the lines are tagged & has correct IDs etc, but also depending on the browser & extensions etc.
So I'm going to do some math, and see how I can solve this. Chances are, something like: saving the number of lines visible on each page, alongside the progress should solve this. So when you open the book, we can then correctly re-calculate the position of all bookmarks & progress, in a much more error-tolerant way. [in theory]
So far we could only reproduce this bug by first uploading a new ebook to Cryptee, reading & bookmarking the book on two different screen sized devices, then trying to load it back on the first device.
If you can confirm this as well, that would be amazing! In the meantime, I'll start doing the math!
astarq
commented
4 years ago Hi again! No problem. It can't hurt to learn! Thanks but it's nothing, I'm just trying to help out with all the extra time during Lockdown! And, naturally, it's pretty great if I can do my bit for the software I use. So kudos for being open source!
Anyway, thanks for explaining ~ makes sense lol
I tried Metamorphosis, since I have it, and the first time it actually loaded my bookmarks and reading position - the two browsers did have the same screen size though. But like you said, it eventually messed up, and when I opened it again on a different browser the reading position wasn't the same and bookmarks don't save.
Oh and... As a side note, when I minimise an ebook using the little bar at the top inside Cryptee, it reloads the book when I open it meaning I lose my position 😬
johnozbay
commented
3 years ago Hey there!
So just to give you an update,
As a side note, when I minimise an ebook using the little bar at the top inside Cryptee, it reloads the book when I open it meaning I lose my position 😬
I think we've managed to address this one. And we also gave the small reader a fixed width and padding, that should fix 10% of the forgetting bookmark / position problems. The rest of the 90% is a lot more tricky. As it turns out we even have to account for not just fonts, paddings, margins etc in the book, but also inline images in some ebooks that render differently depending on the pixel density / resolution of your screen. Making me appreciate the existential purpose of ISO standardization a lot more, and leaving me wishing they sparkled some of that standardization magic on ebooks as well.
We're still looking into finding a way to calculate these better, and will update this thread once I know more.
johnozbay
commented
2 years ago Hi there,
I've got some good news! https://github.com/cryptee/web-client/commit/13fec562257ea35b3fec08d5b3a9ec180e457f78 brings a much better ebook / epub calculator, and renderer. We think this should be fixed now, and your reading progress and bookmarks should be accurately represented and sync'ed across your devices.
Needless to say please feel free to let me know if this issue is still ongoing for you, and we'll be happy to give it another go! Turns out epub on the web is more complicated than we initially estimated.
Thanks a million for filing this and for your patience with all this!
All the very best,
J
Describe the bug The ebook reader never saves where I've read up to. Based on the assumption that it is meant too :) The ebook reader also doesn't save my bookmarks. I even tried making a bookmark then opening the bookmarks tab, but they still aren't shown. (Which makes me murderous when reading real books, nevermind ebooks without page numbers!)😰 To Reproduce Steps to reproduce the behavior: Open any ebook, skip a few pages, save some bookmarks > close and reopen the ebook > observe reading position and saved bookmarks
Expected behavior The place I was last reading and my added bookmarks would be saved
System Information (please complete the following information):
Additional context This happens on any connection, on 4G+, 4G, H+ (by which I mean it might not be connection related)
At first, some of my bookmarks would save and appear on other devices. Now they don't.
I was sure to test the above issues with a number of ebooks, with the same results.