jsoftware / jsource

J engine source mirror
Other
645 stars 91 forks source link

[Feature Request] Native base64 support #113

Closed rdm closed 1 year ago

rdm commented 2 years ago

J has most of the functionality needed to support efficient conversion of base64 values. Two additional native mechanisms would improve the situation:

(*) Explicit implementation of entry of numbers with 64 distinct digits.

Numbers which start with 64b currently work fine, but only 36 digits are supported. Numbers which start with 64B generate an ill-formed error. Base 64 uses numeric characters, lower and upper case as distinct digits and two other characters as digits. Traditionally, these are + and / which are not accepted in J number formats. However, . and _ are accepted in J number formats and could be used in place of + and /.

(*) Support for extended precision numbers in arbitrary bases.

We could allow extended precision representation of the numeric base to indicate that we are generating an extended precision number. For example: 16xbfeedabeadaddadecade.

Complete support for arbitrary base64 numbers requires character substitution, but that could be handled using rplc&'+./_'or something like {&('._',a.)@(('+/',a.)&i.)

rdm commented 2 years ago

Note that if rplc&'+./_' were to be used, 16B digits would ALL be different from 16b digits.

In traditional base64:

A means 0 B means 1 ... J means 9 Z means 25 a means 26 z means 51 0 means 52 9 means 61

This approach is a bit disconcerting to work with.

An alternative approach would be to order digits in ascii order. This would preserve the usual significance of 0..9 but which disrupt hexadecimal because _ would mean 10 . would mean 11 A would mean 12 and so on...

Preserving maximum compatibility with 36bxxxx style numbers and borrowing from base64 practice for the remainder (which is what I had been intending to suggest) would instead give this digit assignment:

0 means 0 .. 9 means 9 a means 10 ... z means 35 A means 36 ... Z means 61 . means 62 _ means 63

With this digit assignment:

B64J=: a.{~;(a.i.'0aA._')(+ i.)&.>10 26 26 1 1
B64W=: a.{~;(a.i.'Aa0+/')(+ i.)&.>26 26 10 1 1

b64decode=:  _. ". '16xB', B64J {~ B64W i. ]

bas64encode would need further support, plausibly a 'B' formatting option for ": and/or 8!:

rdm commented 2 years ago

I guess withdraw this request.

B64W=: a.{~;(a.i.'Aa0+/')(+ i.)&.>26 26 10 1 1

b64dec=:{{64x #. B64W i. y}}

Is more concise.

It lacks error checking, but b64dec=:{{64x #. (i.64) {~ B64W i. y}} would address that issue.

rdm commented 1 year ago

request withdrawn.