chore(common): register .kmx / .kmp with MIME and with magic

[srl295]

• [ ] register .kmx (binary) and .kmp (zipfile) with MIME
• [X] register magic bytes in the appropriate place

.kmp will be challenging because the zip metadata is at the end

[srl295]

The form is here https://www.iana.org/assignments/media-types/media-types.xhtml#image

@mcdurdin some questions:

• I think you should perhaps have your email on the form. Still, I'm glad to get a draft going. In fact, the draft spec could live in the repo.
• Thinking of these: (or vnd.sil? )

extension	mime	Spec Link
.kmx	application/vnd.keyman.kmx	here
.kmp	application/vnd.keyman.kmp+zip	here

• also, update the pages at 'spec link' to actually link to a detailed specification of the format

[mcdurdin]

• KMX does not have formal documentation, should use KMXPlus documentation as a starting point. See #11407.

• KMXPlus documented here: https://github.com/keymanapp/keyman/blob/master/core/src/ldml/C7043_ldml.md, should we copy this to help repo, and give it a better name than 7043?

• KMP documentation includes links to metadata (https://help.keyman.com/developer/current-version/reference/file-types/metadata) which is most of the formal documentation we have.

vnd.keyman is appropriate. We have the keyman.com domain. We don't tend to use the name SIL in any internal names for Keyman, only in branding.

Happy to have my email on the form (perhaps my @keyman.com address).

[srl295]

• .kmp is going to be hard to detect. There's no predictable order of the files. I don't see a lot of prior art for such zipfiles.
• .kmx detection works well. See a patch https://github.com/file/file/compare/master...srl295:keyman?expand=1 (they don't take direct PRs..) - could I get a review of the text there and then I'l submit it to file? It has the MIME type commented out for now but that can be added.

[srl295]

@mcdurdin suggested MIME template

KMX

Your Full Name: ≈

Your Email: ≈

Type Name: application

Subtype Name: 
 - Vendor Tree (vnd. prefix)
 - keyman.kmx

Required Parameters: N/A

Optional Parameters: N/A

Encoding Considerations: binary

Security Considerations:  Does not contain executable content. Application must validate that all data fields are within specification and reject invalid content. 
 ** Or just “the security issues associated with this type have not been assessed” ?

Interoperability Condsiderations: N/A

Published Specification: https://help.keyman.com/developer/current-version/reference/file-types/kmx

Application Usage: This format is produced by Keyman keyboard build tools, and consumed by Keyman keyboard software.

Fragment Identifier: N/A

Restrictions on usage: N/a

Provisional: No

Magic number:  At byte offset zero, “KXTS”

File Extension:  .kmx

Intended Usage: COMMON

[srl295]

and KMP

KMP

Your Full Name: ≈

Your Email: ≈

Type Name: application

Subtype Name: 
 - Vendor Tree (vnd. prefix)
 - keyman.kmp+zip

Required Parameters: N/A

Optional Parameters: N/A

Encoding Considerations: binary

Security Considerations:  As produced by Keyman tools, does not contain executable content. Application must validate that all data fields are within specification and reject invalid content.  As a zipfile, security considerations should be considered as described in https://www.iana.org/assignments/media-types/application/zip

Interoperability Condsiderations: N/A

Published Specification: https://help.keyman.com/developer/current-version/reference/file-types/kmp

Application Usage: This format is produced by Keyman keyboard build tools, and consumed by Keyman keyboard software.

Fragment Identifier: N/A

Restrictions on usage: N/a

Provisional: No

Magic Number: For zipfile: PK\003\004

File Extension:  .kmp

Intended Usage: COMMON

[mcdurdin]

• .kmp is going to be hard to detect. There's no predictable order of the files. I don't see a lot of prior art for such zipfiles.

It looks like it is possible to scan from the end of the file with a negative offset. In which case, we read the EOCD record (we can assume the zip has no file comment), so the record will be 22 bytes long. Something like this seems to work (with a few safety checks along the way):

# Keyman Compiled Package File

# A zip file that contains a file named kmp.json
0   string  PK\003\004
# Check End of central directory record magic
>-22 string  PK\005\006
# Find start of central directory
>>(-6.l)   string   PK\001\002
# Scan central directory for string 'kmp.json', will suffice for a 
# package containing about 150 files
>>>(-6.l)   search/9000   kmp.json   Keyman Compiled Package File
!:ext kmp

I don't think we need to worry about kmp.inf, because that's an obsolete file format, and there will be very few .kmp files that don't have kmp.json.

Prior art on ZIP file content checks: https://github.com/file/file/blob/master/magic/Magdir/msooxml

[mcdurdin]

I think we should hold off on registering MIME types until we have a published spec for .kmx.

• Blocked by #11407

[mcdurdin]

I think we are unblocked now, what do we need to do to move this forward?

[mcdurdin]

MIME type applications submitted.

[mcdurdin]

Keyman mime types: https://github.com/file/file/commit/04edb32ba99ff23ff3969ee606f08dface4e1ff7

[mcdurdin]

• .kmp application/vnd.keyman.kmp+zip has now been registered.
• .kmx application/vnd.keyman.kmx is still in process.

We should add this information to documentation.