search

ODNZSL / nzsl-dictionary-scripts

Scripts needed to support the NZSL mobile applications
MIT License
3 stars 2 forks source link

Add support for a --skip-assets option which will not download and process sign media #19

Closed joshmcarthur closed 9 months ago

joshmcarthur commented 9 months ago

This pull request adds an option to the Signbank extractor so that if --skip-assets is provided, sign media will be recorded in the database, but will not be downloaded or processed (resized, compressed, etc).

This drastically speeds up the import process, taking 20 seconds when I tested locally instead of over an hour. The database files produced are otherwise identical to what a full export produces, they just do not have the supporting assets. These can still be downloaded as required based on data in the database, and in fact this is exactly what NZSL Share and the NZSL Dictionary already do - only the native apps require the supporting media to be exported.

This PR adds support for the option, but does not alter the scheduled release process yet. It's intended that this style of release could be used as a preview, nightly, or on-demand workflow to allow for preview of unreleased content.

joshmcarthur commented 9 months ago

In 7751a07 I added a Github Actions workflow that can be run on-demand to export a Signbank database, and attach it to the release as a build artefact.

joshmcarthur commented 9 months ago

@G-Rath for context, we are wanting to introduce the ability for un-released Signbank data to be previewed in staging environments. This is a building block that should allow a data-only export to be run on demand quite quickly, and then we can either fetch or pass this database to the Dictionary and Share to use.