EBCDIC character support

[XVilka]

Extended Binary Coded Decimal Interchange Code (EBCDIC; /ˈɛbsɪdɪk/) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six-bit binary-coded decimal code used with most of IBM's computer peripherals of the late 1950s and early 1960s. It is supported by various non-IBM platforms, such as Fujitsu-Siemens' BS2000/OSD, OS-IV, MSP, and MSP-EX, the SDS Sigma series, Unisys VS/9, Burroughs MCP and ICL VME.

Rizin should have

• [x] ASCII<->EBCDIC conversion
• [x] Unicode <-> EBCDIC conversion
• [x] EBCDIC strings autodetection

See:

• https://stackoverflow.com/questions/7734275/c-code-to-convert-ebcdic-printables-to-ascii-in-place
• https://github.com/openssl/openssl/blob/master/crypto/ebcdic.c

The pushed code should follow LGPL license (or more permissive ones). There is some existing code in librz/magic/* but it's incomplete and not integrated in the Rizin itself.

The desired location of working with this encoding is librz/util/*

See also https://github.com/rizinorg/rizin/issues/1052

[XVilka]

The most common character sets are:

• https://www.ibm.com/docs/en/xl-fortran-aix/16.1.0?topic=appendix-ascii-ebcdic-character-sets
• https://www.microfocus.com/documentation/enterprise-developer/ed30/VS2017/BKPUPUCHARS001.html
• https://public.support.unisys.com/aseries/docs/clearpath-mcp-18.0/88076385-021/section-000020944.html
• https://www.oreilly.com/library/view/programming-in-cobol400/9780471418467/apb.html

For mapping the particular EBCDIC character set to the UTF-8 you can consult pages like these:

• https://www.compart.com/en/unicode/charsets/EBCDIC-UK
• https://www.compart.com/en/unicode/charsets/EBCDIC-US
• https://www.compart.com/en/unicode/charsets/EBCDIC-ES
• https://www.compart.com/en/unicode/charsets/IBM290

These are lists of all:

• https://www.ibm.com/docs/en/zos/2.3.0?topic=sets-coded-character-sorted-by-ccsid
• https://www.compart.com/en/unicode/search?q=EBCDIC#char-sets

See also the http://www.longpelaexpertise.com.au/ezine/LostinTranslation1.php for some more common EBCDIC character sets. Regarding localized charsets I recommend to cover all US, UK, ES, FR, JP, RU character sets where these machines were the most popular during these times.

See also tests/ in iconv repository: https://git.savannah.gnu.org/gitweb/?p=libiconv.git;a=tree;f=tests;h=931829519771ea1fdc0f102cf9e5aabfd06e170b;hb=HEAD