search

machawk1 / wail

:whale2: Web Archiving Integration Layer: One-Click User Instigated Preservation
https://matkelly.com/wail
MIT License
350 stars 35 forks source link
gui heritrix openwayback pyinstaller python warc wayback web-archiving

readme

WAIL logo

"One-Click User Instigated Preservation"

Web Archiving Integration Layer (WAIL) is a graphical user interface (GUI) atop multiple web archiving tools intended to be used as an easy way for anyone to preserve and replay web pages.

Tools included and accessible through the GUI are Heritrix 3.2.0 and OpenWayback 2.4.0. Support packages include Apache Tomcat, pyinstaller, and MemGator.

WAIL is written in Python and compiled to a native executable using PyInstaller.

NOTE: WAIL has also been reimagined in WAIL-Electron. The reimagined version enables high fidelity web archiving using modern capture and replay tools as well as collection-based archive organization. For natively using conventional institutional-grade Web archiving tools like OpenWayback and Heritrix, the repo where you reside is the place to be.

Installing WAIL

WAIL is an application that runs either on your macOS (.app) or Windows (.exe) system.

To install WAIL:

  1. Download the appropriate package for your file system (.dmg for macOS or .zip for Windows) or see more releases.
  2. Open the package (.dmg or .zip).
  3. Follow the instructions to drag-and-drop the WAIL application to the correct location on your file system.

Alternatively, WAIL can be installed with the homebrew package manager on macOS with:

brew install wail

Running WAIL

This section is intended only to run WAIL from source. To download the compiled application, see the downloads section.

End-user execution is meant to be accessed through the binary file, either WAIL.app on macOS or WAIL.exe on Windows. To run it using Python for testing, run the following from the root of the WAIL source directory:

python ./bundledApps/WAIL.py

Since Wayback and Heritrix configurations rely on absolute paths on the system, checks and interactions with services may not work in debugging mode unless a binary of WAIL (e.g., WAIL.app) currently exists in directory specific to your operating system (see below).

Python is not required to be installed for end-users, just double-click (see above) and go!

Development

To build WAIL from source to a native system executable (.app or .exe), open a terminal or console session and move into the root of the WAIL source directory then:

macOS

sh bundledApps/MAKEFILE.sh

This will create /Applications/WAIL.app on macOS.

Generating App Icons

On macOS, application icons are stored in a .icns file. To generate this:

  1. Create a directory to store images: mkdir wail_blue.iconset
  2. Create a PNG of your high resolution (1024x1024) icon, currently based on /build/icons/wail-blue-logo.psd, name it icon_1024x1024.png and place it in the wail_blue.iconset
  3. In a terminal, cd wail_blue.iconset then run the script to generate the other images needed for an .icns with ../build/resize.sh. There will then be 11 PNGs in the directory.
  4. Navigate to the parent directory with cd ../ and run iconutil -c icns wail_blue.iconset. This will create wail_blue.icns.
  5. Store wail_blue.icns in WAIL/build/icons/ when running building WAIL using MAKEFILE.sh on macOS.

Windows

From the Windows shell:

bundledApps\MAKEFILE.bat

then move the WAIL source directory to the root of your C drive (thus making C:\WAIL).

Linux

Linux support is currently in-development using Docker (see #2). From the root of the WAIL source working directory, run:

docker build -t machawk1/wail .

then

docker container run -d -it --rm -p 5920:5920 --name wail machawk1/wail

To log into the container to view the WAIL interface point a VNC Client to localhost:5920. The container can also be accessed via the command-line with:

docker exec -it wail /bin/bash

Once in the container, run the WAIL executable with:

./WAIL

There still exists some issues to be resolved for #2, as will be evident in the reports on the console.

Problems? Questions?

Please see the Frequently Asked Questions page.

Contact

WAIL is a project of the Web Science and Digital Libraries (WS-DL) Research Group at Old Dominion University (ODU), created by Mat Kelly with additional extensive contributions by John Berlin.

For support e-mail wail@matkelly.com or tweet to us at @machawk1 and/or @WebSciDL.