search

GenderDysphoria / GenderDysphoria.fyi

Site code
Other
224 stars 63 forks source link

Adds WARC generation for offline viewing #95

Open gjvnq opened 2 years ago

gjvnq commented 2 years ago

I added a gulp task to generate WARC files from the website so it can be read offline. These can viewed through WARC players like replayweb.page and they are attached to this pull request to allow the mantainers to evaluate this feature.

There's a minor bug about the sidebar/bookmarks which I'm still investigating (see https://github.com/webrecorder/replayweb.page/issues/99).

Finally, WARCs aren't as good as EPUBs but they are here and don't have as many compatibility quircks as EPUBs.

Attached files: (you will likely need to rename the extensions from .warc.txt to .warc)

gdb-en.warc.txt gdb-zh.warc.txt gdb-de.warc.txt gdb-hu.warc.txt gdb-pl.warc.txt gdb-zh.warc.txt gdb-es.warc.txt gdb-fr.warc.txt

Twipped commented 2 years ago

I've never heard of this format. Is there a meta tag or something that exposes it as available?

gjvnq commented 2 years ago

I don't understand your question.

If you are asking about how the user would get the file, the answer is simple: through a traditional link just like many websites use for PDF and EPUB.

Em qui., 26 de mai. de 2022 11:58, Jocelyn Badgley @.***> escreveu:

I've never heard of this format. Is there a meta tag or something that exposes it as available?

— Reply to this email directly, view it on GitHub https://github.com/GenderDysphoria/GenderDysphoria.fyi/pull/95#issuecomment-1138670553, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXIYHFDITPJWT5QWFQ2J3LVL6GRHANCNFSM5W7PNBSQ . You are receiving this because you authored the thread.Message ID: @.***>

Twipped commented 2 years ago

Ah, okay, I thought it was something that browsers hooked into for offline caching.

So, one small problem with this is that I'm actually trying to get off of gulp. There's also several other changes that I just can't merge at this time without seeing what they do. I'll have to review this more in-depth later.

gjvnq commented 2 years ago

So, one small problem with this is that I'm actually trying to get off of gulp. There's also several other changes that I just can't merge at this time without seeing what they do. I'll have to review this more in-depth later.

I dislike gulp too. What do plan to use as a replacement?

And what can I use for WARC generation? Is it okay if I use something written in Go, Rust or Python?

gjvnq commented 2 years ago

Ah, okay, I thought it was something that browsers hooked into for offline caching.

I can use Manifests (see http://diveintohtml5.info/offline.html) to allow for offline browsing but I think that the usability is subpar.

Twipped commented 2 years ago

What do plan to use as a replacement?

Just plain JS tied together with yargs. The main thing holding me back right now is the AWS publish system. I haven't found anything else that supports pattern based cache control headers. I need to port gulp-awspublish out into a standalone library, but haven't the time to do it.

Twipped commented 2 years ago

Is it okay if I use something written in Go, Rust or Python?

Only if it's something quick and easy to install. Python dependency management is a hell I'd rather not get involved with, and I have zero experience with Go or Rust. I'd much rather it be in JS.

gjvnq commented 2 years ago

Is it okay if I use something written in Go, Rust or Python?

Only if it's something quick and easy to install. Python dependency management is a hell I'd rather not get involved with, and I have zero experience with Go or Rust. I'd much rather it be in JS.

Golang generates static binaries so installing them is pretty easy.

As for the current JS script (offline.js), it uses Gulp very minimally so converting it to work with another tool should be very easy.