Mediawiki - local links to dweb-mirror

[mitra42]

Part of #286 (Mediawiki)

Goal to make it so links in the mediawiki go to the local dweb-mirror rather than back to archive.org

• [x] ArchiveLeaf: fonts see below
• [x] ArchiveLeaf: other links see below (just needs LocalSettings change)
• [x] ArchiveLeaf: links to palmleaf.org - see below
• [x] ArchiveLeaf: manifest fetcher - see below (just needs LocalSettings change)
• [x] ArchiveLeaf: IIIF - see #291 done
• [x] ArchiveLeaf: browsing - done, but see https://github.com/internetarchive/dweb-mirror/issues/289 for further work to build the same "stream" UI that mediawiki uses
• [x] ArchiveMirador;
• [x] ArchiveOrgAuth - be careful, actually want to authenticate to archive.org
• [x] Look at other variables in LocalSettings.php - only ArchiveOrgAuth
• [x] Look for uses of 'https://api.archivelab.org' - only ArchiveLeaf and LocalSettings.php
• [x] Look for IIIF maybe search on iiif.archivelab.org -only ArchiveLeaf and LocalSettings.php
• [x] Make sure any changes make it to palmleaf.org and/or their repos
• [x] Use network control panel to look for other access back to palmleaf.org or archive.org
• [x] Document configuration chosen below
• [x] Testing fails - with prewritten pages -

[mitra42]

ArchiveLeaf - links to Fonts

AL/modules/ext.archiveleaf.common.scss

• [x] Add fonts to mediawiki files
• [x] Make links to fonts rooted
• DEV ARTIFACT IGNORE: transcriber/build/index.html, subset of ext.archiveleaf.common.scss
• DEV ARTIFACT IGNORE: transcriber/public/index.html: src: url("https://archive.org/cors/NotoFonts/NotoSerifBalinese-Regular.woff2") format("woff2"),
• [x] modules/ext.archiveleaf.common.css - built from scss, references to fonts, so can ignore but recheck after rebuild in case missed something

ASKED DAVID:

Can you see any problems adding the fonts listed in ArchiveLeaf/modules/ext.archiveleaf.common.scss to the distribution somehow because loading them from their sources such as https://bali.panlex.org/transcriber/fonts/Vimala.woff2?v22 or https://archive.org/cors/NotoFonts/NotoSerifBalinese-Regular.woff2 is going to be problematic.

FIXED: installtion script ArchiveLeaf/maintenance/offline-fonts fixes it, and is called from maintenance/offline

[mitra42]

ArchiveLeaf other links to archive.org

• AL/extension.json: "ArchiveLeafBaseURL": "https://archive.org" overridden by
• [x] LocalSettings.php should change localhost
• Usage in ArchiveLeaf.class.php*5 specials/SpecialArchiveLeaf.php
  • [x] to stream/IDENTIFIER/.... see #289
  • [x] Check wgArchiveLeafBaseURL usage $wgArchiveLeafBaseURL.'/download/'.$id.'/page/leaf'.$leaf.'_w2000.jpg' (and w400 an w800) - see leaf fetcher below
• DOCUMENTATION ONLY: README.md - just documentation
• [x] modules/ext.archiveleaf.common.css - built from scss, references to fonts, so can ignore but recheck after rebuild
• DEV ARTIFACT transcriber/build/index.html:|ImageBrowse=https://archive.org/stream/tutur-smara-bhuwana/tutur-smara-bhuwana-300ppi#page/n2/mode/1up

[mitra42]

ArchiveLeaf links to palmleaf.org

• OK - README.md
• DERIVATIVE - transcriber/build/static/js/main.3970a6df.chunk.js.map (one item in mediawikiApi)
• DERIVATIVE transcriber/build/index.html (derived from public/index.html - has specific example
• OK ? transcriber/src/App.js - only if NODE_ENV === "development" so prob OK
• CACHE - transcriber/node_modules/.cache/babel-loader/aebdfedfb0a4d41e3eeacdcdda10f9f8.json
• DEV ARTIFACT IGNORE - transcriber/public/index.html - looks odd, specific to one image, maybe built live ?

[mitra42]

Manifest fetcher

• [x] default specified in AL.extension.json
• [x] specified in LocalSettings.php $wgArchiveLeafApiURL = "https://api.archivelab.org"
• DOC ONLY in AL.README.md
• [x] Used in several places where need to use links to DM or DM needs to handle this API too
  • ./extensions/ArchiveLeaf/specials/SpecialArchiveLeaf.php (just checked)
  • [x] ./extensions/ArchiveLeaf/ArchiveLeaf.class.php
  • $wgArchiveLeafApiURL.'/books/'.$id.'/ia_manifest (2 places)
  • [x] fix DM to handle this

Process

• https://api.archivelab.org/books/ArtOfCommunitySecondEdition/ia_manifest odd, cf to
• https://ia802902.us.archive.org/BookReader/BookReaderJSIA.php?id=ArtOfCommunitySecondEdition&itemPath=/28/items/ArtOfCommunitySecondEdition&server=http://dweb.me&format=jsonp&subPrefix=Art_of_Community_Second_Edition&requestUri=/details/ArtOfCommunitySecondEdition
• note the archivelab manifest has a seriously bizarre structure, copied from BookReaderJSON
  • [x] Get format of URL from Mek
• it uses
  • .url -> $remoteUrl -> multiple places
  • .subPrefix -> $subPrefix -> passed to URL, which I discard
  • .leafs -> used as iterator to loop through generating sections of page (just the number used for url)
  • .collection to look up in collection_Map
  • .language -> munged -> $language_code and in .script matching
  • .script -> $script_code
  • .pageWidths[] & pageHeights[] -> $template/FullSize

[mitra42]

Leaf fetcher

• https://archive.org/download/tutur-smara-bhuwana/page/leaf1_w2000.jpg
• https://ia800503.us.archive.org/BookReader/BookReaderImages.php?id=tutur-smara-bhuwana&itemPath=%2F12%2Fitems%2Ftutur-smara-bhuwana&server=ia800503.us.archive.org&page=leaf1_w2000.jpg
• $wgArchiveLeafBaseURL.'/download/'.$id.'/page/leaf'.$leaf.'_w800.jpg
• $wgArchiveLeafBaseURL.'/download/'.$id.'/page/leaf'.$leaf.'_w400.jpg
• FIXED - had to recognize URLs in this form, locate manifest, and scale based on width.

[mitra42]

Testing

• (asked David about URLs and when created)
  • [x] "Image on Archive.org" (below each image) is still link to archive.org
  • [x] Original on Archive.org (top of page) is still link to archive.org
  • [x] Try out current mediawiki release pushed by David 2020-01-07
• [x] urls to iiif are trying https
• [x] urls to iiif and probably others are trying localhost (my mac where browser is) rather than host its running on
  • this is in ArchiveLeaf/transcriber.src
  • [x] see if David knows how to do this in PHP
  • [x] if not ... try "box.net" or "archive"
  • [x] OR proxy through DM to mediawiki and use /
  • David has added config variable to php $wgArchiveLeafArchiveOrgLinkTarget