Include ALAN Assets directly in ISO Encoding

[tajmone]

WIP temp branch: dev-2.2.0-drop-utf8-conv (deleted)

Drop the UTF-8 conversion of ALAN sources and transcripts before their inclusion in ADoc sources:

• [x] StdLib Sources:
  • [x] Delete:
    • [X] extras_src/stdlib2utf8.sh.
    • [X] _assets/utf8/StdLib/.
    • [x] Amend READMEs accordingly:
      • [x] _assets/README.md
      • [x] extras_src/README.md
  • [x] Tweak extras_src/update.sh:
    • [x] Don't create UTF-8 StdLib sources to _assets/utf8/StdLib/.
  • [x] Amend StdLib Man include:: directives accordingly:
    • [x] Fix StdLib inclusion paths to point to original sources.
    • [x] Use the [encoding=iso-8859-1] option.
• [x] StdLib Manual and Tutorials (shared scripts and assets):
  • [x] Tweak extras_src/manual/html-build.sh:
    • [x] Drop $utfDir env var and fix any references to it in:
      • [x] StdLib Manual.
      • [x] Clothing Guide.
  • [x] Tweak extras_src/update.sh:
    • [x] Don't create UTF-8 Version of Alan Sources and Transcripts to _assets/utf8/**.
      • [x] Amend extras_src/README.md accordingly.
      • [x] Delete _assets/utf8/ folder and all references to it.
    • [x] Tweak "Sanitize Game Transcripts" stage so that ADoc formatting can be done directly in original transcript files (see note below).
      • [x] Remove any references to .a3ADocLog extension in READMEs, Git settings, etc.
  • [x] Amend include:: directives in StdLib Man and Clothing Guide accordingly:
    • [x] Fix inclusion paths to point to original sources and transcripts.
    • [x] Use the [encoding=iso-8859-1] option.
  • [x] Amend extras_src/README.md accordingly.
  • [x] Amend _assets/sh/ accordingly (e.g. delete alan2utf8 func=.
• [x] Carry out some more tests to ensure that everything was done properly, and that there are not left-over references to the old system, then squash the changes as a single commit into branch dev-2.2.0.

Transcripts NOTE — We still need the sanitizer script that nicely reformats transcripts into AsciiDoc (player input styles); so we need to decide whether to apply these fixes directly to the original .a3log or if keep using a derivate file (currently using the .a3ADocLog extension inside _assets/utf8/**).

---

Currently all documents in this project that rely on the include:: directives to import external Alan sources and transcripts need to first convert them from ISO-8859-1 to UTF-8.

In 2019 I had submitted a feature request to enable support for other encodings, which was implemented and made its way into stable releases — but hasn't yet been documented! It's usage is as follows:

include::./somefile.alan[encoding=iso-8859-1]

By skipping the UTF-8 conversion step we'll greatly improve the build speed of the whole project.

See how this was done in the Alan-Docs repository:

• https://github.com/alan-if/alan-docs/issues/84

References

• Asciidoctor Manual:
  • §28. 28. Include Directive
• Feature request and related PRs:
  • https://github.com/asciidoctor/asciidoctor/issues/3248
  • https://github.com/asciidoctor/asciidoctor/pull/3419

[tajmone]

@AnssiR66,

this was a rather big change in the toolchain, both in terms of speed improvements in building the docs as well as reducing the toolchain complexity and all the clutter of the intermediate files.

Although I didn't plan to start working again on the StdLib before mid January, I wanted to get this sorted out in view of the changes planned in #103 (i.e. moving the toolchain to a Git submodule that can be shared by all ALAN-related projects, which would reduce drastically future maintenance work).

These changes have been on my TODO list for quite a while, so I finally took a day off to fix it both here and on the Alan-Docs repository:

• https://github.com/alan-if/alan-docs/issues/84

Now that I can finally see it at work, it's definitely a huge improvement, especially in terms of build speed, but also in terms of less scripts and complexity to deal with (which is going to make the next steps toward #103 much easier).

I hope you and @thoni56 are enjoying the season holidays, and wish you a happy 2021!

[AnssiR66]

@tajmone Thanks for taking care of the issue. All the best for 2021 as well!

[thoni56]

Ditto to you guys too!