Epub option to bookreading - see if there is a reader (maybe ask Mark)

[mitra42]

Add EPub option to bookreading (ask Mek if there is a reader) See also page 60 in my notebook for some notes

[mitra42]

• [x] See slack notes from Mek in email
• [x] Find a good public epub example
• [x] Find a good OpenLibrary epub example
• [x] Look on archive.org at example, see how download link integrated - see below
• [x] DM - return metadata see below
• [x] DM - handle failing epubs. ArtOfCommunitySecondEdition.epub. SEE BELOW re this fail
• [x] DM - allow epubs to download: e.g. LasCrucesSobreElAguaJoaquinGallegosLara
• [x] Make crawler grab epub - add to minimumFiles
• [x] Add size in files.json if have file and downloaded
• [x] How to display epub in bookreader or as alternative @IA
• [ ] Think thru any issues related to WorldReader https://github.com/internetarchive/dweb-mirror/issues/245
• Check notebook pg 60 for any other notes
• [ ] Ask Dan (slack.db48x) about algorithm for it being a public ebook (slacked Dec27)
• [ ] Note epub generator is buggy - see https://github.com/internetarchive/epub3/issues/39 for known example @IA
• [x] move crawl.opts.crawlEpub to configDefaults.yaml and set to true
• [ ] Currently failing on http://localhost:4244/epubreader/index.html?bookPath=/download/ArtOfCommunitySecondEdition/ArtOfCommunitySecondEdition.epub not finding JS

CLEANUP (fix or create new issues when above tackled)

• [x] integrate reachable into test in DownloadableOptions rather than in AnchorDownload
• [ ] https://dweb.me/arc/archive.org/metadata/ArtOfCommunitySecondEdition/ArtOfCommunitySecondEdition.epub should work on gateway, (though not currently being used)
• [x] Find out what the "Reader format available" hook is that Safari (at least on iPad) uses, build into page if/when ePub available, and ideally backport to archive.org
  • This doesnt quite make sense, the Reader view is triggered by the HTML content, not by a separate download
• [ ] Embed epubreader into theatre, but allow it to go full-screen, maybe just grab inner part of the reader's index.html

[mitra42]

• LasCrucesSobreElAguaJoaquinGallegosLara localhost download archive.org download working
• The_Woodworkers_Studio_Handbook_by_Jim_Whitman localhost details download archive.org download
• ArtOfCommunitySecondEdition localhost download archive.org details download (fails)

[mitra42]

Brenton said: Yeah, we have no epub compatibility in BookReader. If we wanted to add that functionality to our site (whether in the context of BookReader or not), we could use ReadiumJS. and then We haven’t so far really researched integration possibilities, but if it’s easy..

[mitra42]

I can't figure out Readium.js never mind how to integrate it with Bookreader, an alternative would be epub.js https://github.com/futurepress/epubjs-reader which from view-source:https://futurepress.github.io/epubjs-reader/index.html looks much simpler as a stand alone e-reader

[mitra42]

Working on download link ...

• [x] Fix https://github.com/internetarchive/dweb-archive/issues/139 at same time
• [x] Integrate straightforward epub download into dweb-archive
• [x] Write heuristic that decides if have an epub
• [x] In ArchiveItem. loadFromMetadataAPI(metaapi) - Add formulaic epub and kindle (.mobi) files if matches test

[mitra42]

• [x] http://localhost:4244/arc/archive.org/metadata/ArtOfCommunitySecondEdition/ArtOfCommunitySecondEdition.epub fails
  • [x] Looks like http://localhost:4244/arc/archive.org/metadata/ArtOfCommunitySecondEdition fails on DM
  • [x] Bad follow on to this failure - crashes with multiple lock errors and hten a throw new ReadError('get() requires a key argument')
  • Why is it crashing after lock failures probably hits count out
  • Why is lock getting set ?
    • doing a get with no value - now catch that and throw error
    • doing a hashstore lookup even with no digest - fixed
  • Why is it even doing a fetch of https://dweb.me/arc/archive.org/metadata/ArtOfCommunitySecondEdition/ArtOfCommunitySecondEdition.epub
    • Error no longer happening - not getting this info - now above errors fixed
  • [x] Why is that URL failing on gateway

STR rm /Volumes/Transcend/archiveorg/.hashStore.sha1.relfilepath/LOCK curl -o/tmp/foo http://localhost:4244/arc/archive.org/metadata/ArtOfCommunitySecondEdition

[mitra42]

...replaced by below...

[mitra42]

FAILING EPUB - FIX Plan ... top to bottom

• [x] Make sure current error tracks back to top rather than triggering unrelated failures.
• [x] make sure error returned
• [ ] work on issue so it gets fixed https://github.com/internetarchive/epub3/issues/39

[mitra42]

GOOD EPUB DOWNLOAD

• localhost goes to dweb.me which fails. Should go to: archive.org
• DM catch *epub > DM.streamArchiveFile

• streamArchiveFile does fetch_metadata ok archiveitem.files[16].metadata.name = *epub

• archivefile.cacheAndOrStream

• AF.urls

• AF.httpUrl which generates dweb.me url

• passes url to MirrorFS.cacheAndStream FAIL

• [x] redirect to archive.org, via new upstreamPrefix() function
• [x] check cached

[mitra42]

To Display Switch DM and DA to epub branches, restart server and rebuild webpack URL is something like http://localhost:4244/epubreader/index.html?bookPath=/download/potbouille02zola/potbouille02zola.epub http://localhost:4244/epubreader/index.html?bookPath=/download/LasCrucesSobreElAguaJoaquinGallegosLara/LasCrucesSobreElAguaJoaquinGallegosLara.epub

Displaying in UI

• [x] Added to dweb-mirror#epub and dweb-archive#epub
• [x] need link against EPUB download for viewing it
• [x] merge into master

[mitra42]

• [ ]Figure out algorithm for a) its an epub and b) doesnt require lending ... See chat with db48x.
• Rule texts + file:format=Abbyy GZ
• Rule member of xxx means can’t lend

https://github.com/internetarchive/dweb-mirror/issues/240 |---|Epub|Access|Issues| |ArtOfCommunitySecondEdition|y|y|Getting not allowed - retest| |potbouille02zola|