search

rust-lang / mdBook

Create book from markdown files. Like Gitbook but implemented in Rust
https://rust-lang.github.io/mdBook/
Mozilla Public License 2.0
18.09k stars 1.63k forks source link

Offline support #546

Open sorin-davidoi opened 6 years ago

sorin-davidoi commented 6 years ago

We could leverage the Service Worker API to make the book available offline.

Brief outline:

The main question is - would this be worth it? If I understand correctly, The Rust Book is bundled with Rust (or packaged in some distributions), so most of the people interested in it you have it locally. But then again, it might be nice to provide this regardless.

Michael-F-Bryan commented 6 years ago

How would an offline version work? My understanding is that it'd have to cache all the pages in the book and if the user goes offline we serve pages up from the cache. That'd involve the user effectively trying to download everything the moment they visit a book and the service worker starts up in the background, wouldn't it? That's probably not a good idea for something as big as The Book or Rust By Example...

My opinion is that if it's easy enough to implement and maintain, then I'm all for it. I've (briefly) played around with Service Workers and found they can have issues with updating and caching. That could just be me being a noob though.

sorin-davidoi commented 6 years ago

We could make it configurable, such that the user makes an active choice regarding which chapter should be available offline. My idea would be to don't cache anything by default and show a kind of unobtrusive action the first time the user loads the page, prompting to make it available offline (if its small enough, otherwise just the first few chapters).

Michael-F-Bryan commented 6 years ago

We could make it configurable, such that the user makes an active choice regarding which chapter should be available offline.

I don't know if this would work in practice. People reading a book probably won't want to spend a while configuring how they read the book, and I'm quite conscious of how many buttons and options we add to the UI.

sorin-davidoi commented 6 years ago

Yes, I guess you're right. Could we then try to cache as much as possible?

Michael-F-Bryan commented 6 years ago

I just checked and the user guide itself is already 1.5M (644K when compressed), so storing the entire book may be a bit heavy. If you're only counting the HTML though, it drops to 284k (uncompressed).

What are your thoughts?

sorin-davidoi commented 6 years ago

Well, it has to be HTML + JS, but I think that it is perfectly reasonable - this PWA caches entire videos in the Service Worker, so I don't think we should hit major roadblocks.

We could even compute, at compile time, how much we need to store and show that to the user. I imagine a small banner at the bottom, something like: "Want to make available offline (2.45 MB)?"

Michael-F-Bryan commented 6 years ago

yeah I guess that's feasible, although we'd need to make sure the notification collapses down or hides when the user isn't interacting with it.

How does cache invalidation work? At the moment your typical dev experience is to run mdbook serve on your computer and then it'll livereload as you're editing. If the service worker gets in the way of that livereload, or if it tries to download the entire book a dozen times a minute (because you typed a couple words, hit save, then typed some more, all in quick succession), then that's gonna be a problem...

We can also use book.toml to tell the HTML renderer whether to include the service worker.

sorin-davidoi commented 6 years ago

Service Workers are usually disabled in the development environment to avoid the issues you've mentioned. Not sure how that would work in this case.

ghost commented 6 years ago

This is a great idea. The service work acts as a cacher and updater from a central place like a s3 bucket, etc.

Does stdweb provide Service Worker scaffolding ? https://github.com/koute/stdweb

jasonwilliams commented 6 years ago

@Michael-F-Bryan having something like The Book available offline is exactly the sort of thing service workers were built for. We should certainly do this. we could either do it automatically (users can easily clear their cache) or we can add a “store for offline” button. I don’t think 1.5M is heavy but that’s subjective.

@sorin-davidoi what’s the update on your PR?

Linking: https://github.com/rust-lang/rust/issues/20866#issuecomment-388191907

Michael-F-Bryan commented 6 years ago

@jasonwilliams I agree that it'd be super useful and would be another step towards making mdbook more convenient to work with offline.

We just need to keep in mind that a book isn't always backed by a server, so something like mdbook build --open should continue to work and not care about service workers. From memory there are some general issues with running JavaScript from a file://... URL and using it to fetch documents from the file system.

I support the idea, we just need to be mindful of how it's implemented so it'll integrate well with the current workflow. Off the top of my head, this would require we

sorin-davidoi commented 6 years ago

@jasonwilliams No updates, there were some issues that needed to be handled before implementing this (see https://github.com/rust-lang-nursery/mdBook/pull/571).

jasonwilliams commented 5 years ago

@sorin-davidoi what are the issues? Is there anything i can pick up from this to carry on the work you were doing? It looks like this has just come to a stop

sorin-davidoi commented 5 years ago

@jasonwilliams I think the main issue was the use of external dependencies, since they paths need to be hardcoded in multiple places (and it also makes it much harder to compute their revision). Not sure how valid this still is since I'm stopped watching this project a while back.

jasonwilliams commented 5 years ago

@sorin-davidoi thanks, why is the paths changing a problem? the workbox tool you use seems to accept regular expressions, could you not make a regex based on the version? Or does this not solve the problem?

I can start by bringing the external deps into the project locally, but is this a hard copy and paste or a build function which can fetch these deps? (like npm)

sorin-davidoi commented 5 years ago

the workbox tool you use seems to accept regular expressions, could you not make a regex based on the version?

I guess that could work.

I can start by bringing the external deps into the project locally, but is this a hard copy and paste or a build function which can fetch these deps? (like npm)

Probably a hard copy, but it would be nice to hear the opinion of one of the maintainers.

jasonwilliams commented 5 years ago

Assuming the regex works, were there any other blockers to getting this working?

sorin-davidoi commented 5 years ago

Ideally you would want to have a list of all these assets upfront, so they can be prefetched. If you don't have that you risk running into the following situation:

jasonwilliams commented 5 years ago

So long as we've identified all the assets (which i think you have in the PR), we should be ok Is there a good directory right now to put all of these assets? i'm guessing in theme

jasonwilliams commented 4 years ago

Help Needed!

https://github.com/rust-lang/mdBook/pull/1000 seems to be working pretty well from what i can see so far, I just need some feedback now. The PR allows the rust book to be read offline, new changes will still take affect.

You can just navigate to https://jason-williams.co.uk/book/ and try it out, then leave any feedback in the issue above.

sanmai-NL commented 3 years ago

@jasonwilliams I‘d like to help you. I use mdBook in higher education. Even today, we had a problem with our harmless github.io subdomain being blocked by Cisco Umbrella. Also, education administrators rightfully dislike educational content being dependent on (uncontracted) service providers’ availability. Having our mdBook availably offline, this time from its canonical URL rather than a local copy (which can get out of date, which is hard to manage for hundreds of students at the same time), would be a great feature.

image

nihaals commented 3 years ago

github.io seems to be a common thing to be blocked even outside Cisco

sanmai-NL commented 3 years ago

@nihaals: yet at the same time it is a common and convenient publishing domain ...

By the way, this is duplicate of #463.

nihaals commented 3 years ago

The way you can help is by reading from https://github.com/rust-lang/mdBook/pull/1000#issuecomment-607861247 and giving any feedback or ideas (or possibly implementing the algorithm I wrote which can just be updated if we end up using only one of the config options)