CLI interface to libtifiles for portable file manipulation

[debrouxl]

Making a CLI interface on top of libtifiles, to provide portable, native code functionality for manipulating computer files suitable for calculators, is a very old idea, and I should have pursued it earlier...

Scattered, partial implementations of the concept can be found in e.g.:

• Benjamin Moody's tipack: https://www.ticalc.org/archives/files/fileinfo/399/39967.html ;
• Jonimus' tifileutils: https://github.com/Jonimoose/tifileutils , https://www.cemetech.net/forum/viewtopic.php?t=4917 ;
• TICT TI-68k Developer Utilities nearly all made by Thomas Nussbaumer in C, for which GCC4TI is the new upstream : https://github.com/debrouxl/gcc4ti/tree/next/trunk/tigcc/tools . These aren't based on libtifiles.
• keoni29's Python ti83f: https://bitbucket.org/keoni29/ti83f .
• PatrickD's var8x: https://www.ticalc.org/archives/files/fileinfo/441/44105.html .
• MateoC's convbin: https://github.com/mateoconlechuga/convbin

Also, to gather all of the links in a single place, let's mention better CLI front-ends for libticalcs, namely Benjamin Moody's titools and Jonimus' fork containing tidump: https://www.ticalc.org/archives/files/fileinfo/430/43083.html , https://github.com/Jonimoose/TITools .

A unified version of a subset of the file manipulation utilities' functionality could be fulfilled by a single tifileutil (name TBD) tool with scriptable interface, ala gdbmtool / tdbtool (pure CLI args version at first). I think that it should live in the tilibs tree, alongside the other top-level projects, so that it can be used e.g. when building libticalcs' ROM dumpers - that's the trigger for writing this issue.

Draft proposal gathering functionality found in other tools, as well as several new items:

• tifileutil convert [...]: converting to other formats;
  • [x] P1 conversion to C array, for the immediate use case of replacing tf2hex;
  • [ ] P2 binary <-> hex conversion for TI-Z80 PPRGM ("squishing" / "unsquishing");
  • [ ] P2 binary <-> exec string conversion for TI-68k ASM programs. Found in various programs and ttasm2exec.pl .
  • [ ] P2 other formats.
• tifileutil wrap: wrapping binary into variables of various types.
  • [x] P1 simple mode, ala tipack: producing computer metadata for pre-made variable data of just about any type of variable, including complex numbers / lists / variables and TI-(e)Z80 protected programs;
  • [ ] P2 envelope mode, ala ttbin2oth, ttbin2str, ttbin2txt and others: producing an envelope of TI-68k "other" type or TI-68k string type for generic data, and TI-68k text type for text containing newlines, then producing computer metadata wrapping that envelope. The string type also exists on the TI-Z80 & TI-eZ80 series; other potentially relevant types include picture type (TI-68k, TI-Z80, TI-eZ80), image type (color TI-Z80, TI-eZ80), some Python appvars (TI-eZ80 with Python).
• tifileutil unwrap (tistrip-inspired, replacement for ttstrip, 8xpconv - https://codewalr.us/index.php?topic=1027.0 - or convbin).
  • [ ] P1 simple mode, ala ttstrip: stripping the header and trailing checksum (if any);
  • [ ] P2 deep mode, e.g. with a -d switch: performing deeper stripping of the envelope for some file types, e.g. removing the initial token bytes for TI-Z80 and TI-eZ80 protected programs, TI-eZ80 Python appvars, TI-68k strings / other data / ASM programs.
• tifileutil metadata (inspired by lots of tools): printing or setting (a subset of) metadata - name, attributes, checksum with mismatch information, etc. - for a computer variable, in one fell swoop. For best results, the arguments of wrap and metadata commands should be maximally compatible... and in fact, in order to avoid significant code duplication, it may well happen that the modification functionality of metadata be internally implemented as unwrap + wrap with different parameters. Metadata writing and reading functionality could be useful.
  • [ ] P1 print mode: printing (a subset of) information about a file (ttinfo-inspired), including looking into group-type formats if requested. Found in tifileutils: calls to tifiles_*_display* on groups.
  • [ ] metadata writing and reading mode.
  • [ ] P1 name & folder: get and set on-calc variable and folder name, for both single variables (ttsetname-inspired) and group-type variables (tifileutils-inspired). Also found in convbin.
  • [ ] type & model: getting and setting the model and "variable type for linking" byte (e.g. 0x21 / 33 for TI-68k ASM programs). Found in tipack, but both wrap and metadata commands need a generalized version of this functionality to produce variable types which do not have an official extension, and metadata needs a way to specify the model (e.g. for implementing a generalized version of ttbin2bin, which merely switches between **TI89** and **TI92P* according to the file extension);
  • [ ] comment: get and set file comment. Found in tifileutils and tipack.
  • [ ] attributes: get and set file attributes: the archived flag (TI-(e)Z80, TI-68k) and the locked flag (TI-68k). Found in tifileutils (timod) and convbin.
  • [ ] version: get and set version (TI-(e)Z80).
  • [ ] P1 checksum: detect and report but don't fix checksum mismatches, or compute trailing checksum based on content specified to be correct (ttchecksum-inspired).
• tifileutil group: dealing with group-type files. Found in tifileutils.
  • [ ] P2 tifileutil group -c: creating group or tigroup;
  • [ ] P2 tifileutil group -a: appending to group or tigroup;
  • [ ] P2 tifileutil group -x: extracting from group or tigroup;
• P3 tifileutil ???: some form of front-end to the semi-recently added certificate format parsing functions ?? Proper recursive dumps require lots of platform-specific knowledge, though.

tipack CLI

Usage: tipack [OPTIONS] [FILE | -]
Where OPTIONS may include:
 -o FILE:     output result to FILE
 -n NAME:     set on-calc variable name to NAME
 -t TYPE:     set variable type to TYPE (e.g. 82p)
 -c COMMENT:  set file comment (strftime format string)
 -p:          protect program
 -C:          number/list/matrix is complex
 -a:          send files to archive
 -r:          raw mode (no length bytes)
 -v:          be verbose

Implementation notes:

• unlike a decade ago, adding new dependencies onto Glib goes in the wrong direction. I'm trying to reduce dependency on Glib, in order to ease targeting e.g. WebAssembly through Emscripten.

[debrouxl]

tifileutil with wrap and dump sub-commands are now live on the experimental2 branch, used by the libticalcs build when rebuilding the ROM dumpers.