Static generated content for long time storage

MachineVisionUiB / machinevision

We are developing a database to map and interpret the representations and uses of machine vision technologies in digital art, computer games and narratives such as science fiction novels, movies and creepypasta.

http://uib.no/en/machinevision

4 stars 0 forks source link

Static generated content for long time storage #153

Open steinmb opened 2 years ago

steinmb commented 2 years ago

Split out from issue #114. A static copy of the dynamic generated Machine Vision database would ease the long time storage of the project together with the different data exports and full database dump.

Challenges with log time storage of databases

TBD

Drawbacks

Search as we have it now will not work.
No access control. Logged in users vs not logged in users. This will create challenges around the media/screenshots copy right issues since they contain data that is not public.

Scope of work

Test out static generator system for Machine vision database.

steinmb commented 2 years ago

Currently testing out generating a static usable version of the site.

Screenshot 2022-02-08 at 15 55 42

Takes a while to run. Plan is to push it to static.machine-vision.no as soon as the init build is done.

steinmb commented 2 years ago

I let is run for about 48hours but it still was not done. What it finished looked very promising.

Performance adjustment aka, allowing a full site generation to finish

I think stopping it from hitting the data exports is a good start. When it do it will grind to a halt for potentially hours.
I think it generate pages with the redirects (ERROR 301) based on our change aliases. Those we do not need and we should make sure to drop all of them before building.
Paragraphs is OK but orphan paragraphs lead to a lot of processing. We should make sure all orphans are gone before running.
Development settings disable all JS/CSS aggregating and caching. This should be re-enabled before running.