Better codepage support

[ali1234]

Codepage support currently only works for Latin/English and partial Cyrillic support. Latter has to be manually selected and doesn't work for console output - only HTML.

My understanding of how it works:

• For a given page there are two G0 charsets, and one G2 charset.
• ESC switches between the two G0 charsets. It does not affect G2 characters.
• (packets X/28/0 Format 1 or X/28/4) and (packets M/29/0 or M/29/4) define the charsets used by a page or magazine respectively.
• In absence of those packets, defaults are programmed into the decoder? (ie local code of practice.)
• Control bits in the page header further modify the charsets. Exact meaning depends on the current selected charsets or local code of practice. These bits are also duplicated in the X/28 and M/29 packets?
  • There are five basic regions: Latin, Greek, Cyrillic, Arabic, Hebrew
  • Latin and Cyrillic have national options. For Latin these redefine a fixed subset of characters. For Cyrillic they redefine different characters.
  • Hebrew doesn't have its own G2 charset.

[ali1234]

It will be difficult to handle X/28 and M/29 packets in the current codebase because there is no state for the Parser object. That is, it doesn't know what packets have been processed before or after the current one. For HTML it is less of a problem because it operates on whole pages only, but it still can't handle M/29 - but this can be implemented in Service fairly easily.

Currently that means for console output we are limited to allowing the user to select a "local code of practice" which is used for all packets. That seems fairly close to what level 1 does, and we don't really support higher levels anyway.

• [ ] To help the user do this, the console output should decode X/28 and M/29 packets when printing them.