Make Puppy CLI more powerful

This is a general request:

Let's make the Puppy CLI easier and more powerful... A range of command-line programs for setting up the system, with:

consistent naming:
- puppy-installer
- puppy-wifi
- puppy-bluetooth
- puppy-samba
- puppy-xorg
- puppy-wm
- puppy-rescue (fix windows registry, remove passwords, etc)
- puppy-help (custom menu, uses tldr, man, howdoi, cheat.sh, etc)
consistent options (--help, --version, etc)
consistent behaviours (use gettext for translations, etc),
consistent styles (colours, layouts, etc)
... then a nicee wrapper script for them:

puppy-config           # load a main menu, list all config tools, let user choose one

puppy-config wifi       # loads puppy-wifi

puppy-config --help      # describes each tool

puppy-config wifi --help   # describe help options (same as puppy wifi --help)

Programs with a good CLI make it easier to:

document how to do things in Puppy
use Puppy without X
use copy & paste code examples
use Puppy as a chroot
use Puppy over ssh
create CGI web frontends running on localhost for CLI programs
use Puppy as a home server
follow UNIX philosophy (combine & pipe commands)
build new programs upon pre-existing commands
make "noarch" packages for all Puppies
distribute and build programs and their packages from source
support more hardware
be a better rescue OS
make better GUIs easier, faster
de-couple program from its interfaces...(ncurses, GTK, HTML, etc)
create monolithic "control panel" GUIs

Any program that requires a GUI will have most of these problems:

not "noarch" packages, need compiling for each architecture
requires working X
requires a working window manager
often requires working mouse or touchpad
harder to use over a network (and slow... ssh + x11 forwarding, TeamViewer, etc)
much harder to document how to use it
much harder to share re-producible "solutions", "recipes", "examples", etc
harder to document all features in Wikis, READMEs, etc
cannot be used programatically (except with xdotool etc)
usually cannot have web-based interface
does not follow UNIX philosophy (can't be used as part of anything else)
usually much, much harder to compile and package it up
has these deps (or similar):

# ldd `which gtkdialog`
  linux-gate.so.1 (0xb5a96000)
  libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0xb559d000)
  libgdk-x11-2.0.so.0 => /usr/lib/libgdk-x11-2.0.so.0 (0xb54dc000)
  libpangocairo-1.0.so.0 => /usr/lib/libpangocairo-1.0.so.0 (0xb54ce000)
  libatk-1.0.so.0 => /usr/lib/libatk-1.0.so.0 (0xb54a7000)
  libcairo.so.2 => /usr/lib/libcairo.so.2 (0xb5366000)
  libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0xb533b000)
  libgio-2.0.so.0 => /usr/lib/libgio-2.0.so.0 (0xb5167000)
  libpangoft2-1.0.so.0 => /usr/lib/libpangoft2-1.0.so.0 (0xb514f000)
  libpango-1.0.so.0 => /usr/lib/libpango-1.0.so.0 (0xb50fc000)
  libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb50a0000)
  libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb4f76000)
  libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0xb4f33000)
  libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xb4e7f000)
  libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0xb4e7c000)
  libglade-2.0.so.0 => /usr/lib/libglade-2.0.so.0 (0xb4e61000)
  libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb4c80000)
  libvte.so.9 => /usr/lib/libvte.so.9 (0xb4bde000)
  libX11.so.6 => /usr/lib/libX11.so.6 (0xb4a90000)
  libXext.so.6 => /usr/lib/libXext.so.6 (0xb4a7b000)
  libpthread.so.0 => /lib/libpthread.so.0 (0xb4a5c000)
  libc.so.6 => /lib/libc.so.6 (0xb48a5000)
  libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb48a0000)
  libXcomposite.so.1 => /usr/lib/libXcomposite.so.1 (0xb489c000)
  libXdamage.so.1 => /usr/lib/libXdamage.so.1 (0xb4898000)
  libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0xb4891000)
  libm.so.6 => /lib/libm.so.6 (0xb483c000)
  libXrender.so.1 => /usr/lib/libXrender.so.1 (0xb4830000)
  libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0xb482c000)
  libXi.so.6 => /usr/lib/libXi.so.6 (0xb4819000)
  libXrandr.so.2 => /usr/lib/libXrandr.so.2 (0xb480c000)
  libXcursor.so.1 => /usr/lib/libXcursor.so.1 (0xb47fe000)
  libpixman-1.so.0 => /usr/lib/libpixman-1.so.0 (0xb474e000)
  libpng16.so.16 => /usr/lib/libpng16.so.16 (0xb4714000)
  libxcb-shm.so.0 => /usr/lib/libxcb-shm.so.0 (0xb4710000)
  libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb46e4000)
  libxcb-render.so.0 => /usr/lib/libxcb-render.so.0 (0xb46d5000)
  libz.so.1 => /lib/libz.so.1 (0xb46ba000)
  librt.so.1 => /lib/librt.so.1 (0xb46b1000)
  libffi.so.6 => /usr/lib/libffi.so.6 (0xb46a8000)
  libdl.so.2 => /lib/libdl.so.2 (0xb46a3000)
  libpcre.so.3 => /lib/libpcre.so.3 (0xb462a000)
  libresolv.so.2 => /lib/libresolv.so.2 (0xb4610000)
  libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0xb4573000)
  libthai.so.0 => /usr/lib/libthai.so.0 (0xb4568000)
  libexpat.so.1 => /lib/libexpat.so.1 (0xb453e000)
  libicui18n.so.57 => /usr/lib/libicui18n.so.57 (0xb42a1000)
  libicuuc.so.57 => /usr/lib/libicuuc.so.57 (0xb40f3000)
  libicudata.so.57 => /usr/lib/libicudata.so.57 (0xb2875000)
  liblzma.so.5 => /lib/liblzma.so.5 (0xb2849000)
  libncurses.so.5 => /lib/libncurses.so.5 (0xb2823000)
  libtinfo.so.5 => /lib/libtinfo.so.5 (0xb2800000)
  /lib/ld-linux.so.2 (0xb5a97000)
  libXau.so.6 => /usr/lib/libXau.so.6 (0xb27fa000)
  libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb27f3000)
  libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0xb27c4000)
  libdatrie.so.1 => /usr/lib/libdatrie.so.1 (0xb27ba000)
  libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb2640000)
  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb2622000)
  libbsd.so.0 => /lib/libbsd.so.0 (0xb2606000)

In contrast, CLI programs are:

usually "noarch" packages
usually just scripts - easy to edit, simple code (compared to C, C++, Gtk, etc)
very few deps
doesn't require X
can still be very easy to use
can be the backend of multiple GUIs or graphical programs (re-usable)

Create a `split_cli_from_gui` branch

Let's start a branch in which the sole purpose is to separate Puppy programs from their GUIs, leaving separate CLI and GUI versions of each.

A de-coupled, nicely separated CLI + GUI program has all the benefits and none of the drawbacks.

We can tackle one program at a time, and merge them into testing as each one is done.

A PR for each program

Each program done should get its own PR, and each PR should:

Identify one program that is GUI only (and cannot be used without X) or very complex to use in terminal

Lets assume it's called programX.

[ ] create CLI script for programX
[ ] update the existing programX GUI to programX-gui, make it use the CLI script
[ ] include a (simple/demo) ncurses UI
[ ] in PR docs: show CLI usage, the programs help output, and some usage examples

Start with important system and setup programs which would greatly improve the non-X experience of Puppy, if they had easy CLI interfaces..

Creating command line interfaces

We could use a barebones, generic shell script template that parses command line arguments options robustly, and use that as a basis for our CLI scripts.

It could give us the important things "out of the box" and be a good "hello world" or template, setup by default to provide:

a good package structure (file/dir layout, config files/assets in right place, etc)
good parsing of $1, $2, etc
parsing of ENV vars
loading and parsing of a config file
a --help option
a --version option
a --config <file> option
gettext examples

Programs that could do with a good CLI:

the various "Puppy Installer" programs
Quick Setup (First run)
Pmount
setup devices: USB, Bluetooth, Audio, Wifi, ethernet, CUPs, SAMBA, etc
package and repo management (Pkg has this covered, I guess)
BootManager
ffconvert
the Peasy** programs
Pmusic
etc

Other benefits

We would end up with a much more powerful no X puppy, that was easier to use for beginners.

We would have a collection of programs that could be combined via a nice ncurses multi-menu .. A non X replacement of the JWM menu, for example.

We could then clean up, and cut down some GtkDialog (or yad, etc) GUI code

The best way to start is to propose the interface in a PR, to iron out any usage or user experience issues..

For example, someone might propose a CLI advert blocker:

# /usr/sbin/advert-blocker --help

Commands:

 list                list all block list providers

 install <provider>  add the blocklist to /etc/hosts

 remove  <provider>  remove the blockist from /etc/hosts

Options:

 --help              show this help message

 --version           print program version

After seeing the intended CLI described above in a PR, people could suggest fixes, improvements, new features, easier syntax, etc.

Then it can be built, reviewed etc, before being merged..

Use pre-existing programs

One thing we can do is to document, test and use pre-existing command line programs.

There are many good Python, Perl and Bash scripts which would work "out of the box" and provide very user-friendly ways of doing difficult things.

For example, colordiff (replacement for diff,more colourful.. also see diff-so-fancy)

Or ranger, is a very powerful file manager for the terminal, which could be included in the main SFS, or devx (for power users).. It can preview all kinds of files in the terminal, even PDFs, etc.

Or nnn, a file manger in the terminal, lighter than ranger.

There are many "Awesome" lists on Github showcasing these programs, and many scattered around the forum.

Which leads me to another suggestion: https://github.com/puppylinux-woof-CE/woof-CE/issues/1628

Nicer terminal settings:

Add proper support for skipping words, lines, etc in /etc/inputrc:

# Word navigation

# Ctrl-Left/Right:  skip whole words
"\eOc": forward-word
"\eOd": backward-word

# Other potential entries for Ctrl-Left/Right
"\e5c":  forward-word
"\5d":  backward-word
"\e[5C":   forward-word
"\e[5D":   backward-word
"\e[1;5C": forward-word
"\e[1;5D": backward-word
"\e\e[C":  forward-word
"\e\e[D":  backward-word
"\e\e[5C":   forward-word
"\e\e[5D":   backward-word
"\e\e[1;5C": forward-word
"\e\e[1;5D": backward-word

# Alt-left, Alt-right:  skip whole words
"\e[1;3C": forward-word
"\e[1;3D": backward-word

# Other potential entries for Alt-Left/Right
"\e3c": forward-word
"\e3d": backward-word
"\e[3C": forward-word
"\e[3D": backward-word
"\e\e[3C": forward-word
"\e\e[3D": backward-word
"\e\e[1;3C": forward-word
"\e\e[1;3D": backward-word

# for linux console
"\e[5~": beginning-of-history
"\e[6~": end-of-history
"\e[3~": delete-char
"\e[2~": quoted-insert
"\e[1~": beginning-of-line
"\e[4~": end-of-line

# for xterm
"\eOH": beginning-of-line
"\eOF": end-of-line

Improve the `Ctrl-X,<key>` macros:

# BASH keyboard macros (Ctrl-x,<something>)
$if Bash

# Ctrl+x, then p
# Edit the path
"\C-xp": "\C-a\C-kexport PATH=${PATH}\e\C-e\C-a\ef\ef\C-f"

# Ctrl-x,then l
# Edit the library path
"\C-xl": "export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}\e\C-e\C-a\ef\C-f"

# Ctrl-x,then c
# Edit the cd path
"\C-xc": "export CDPATH=${CDPATH}\e\C-e\C-a\ef\C-f"

# Ctrl-x, then "
# prepare to type a quoted word:
# insert open and close double quotes
# and move to just after the open quote
"\C-x\"": "\"\"\C-b"

# Ctrl-x,q
# Quote the current or previous word
"\C-xq": "\eb\"\ef\""

# Ctrl-x,r
# Refresh the line
"\C-xr": redraw-current-line

# Alt-Backspace
# delete word before cursor
"\e\e\C-h": backward-kill-word

# Alt-Delete
# delete word after cursor. (Alt+D by default)
"\e[3;3~": kill-word
"\e\e[3;3~": kill-word

$endif

Friendlier `PS1`

terminal-with-icons

I'm not suggesting we copy mine, but this screenshot demos the following features:

lots of info in the prompt, especially about current directory

^ that's the main benefit.. but also:

coloured icons in terminal using a modified Deja Vu Sans Mono
Git aware ls, with icons and colours.. uses exa.. very large binary :/
a custom cd command that runs ls && source .env after each cd (and more)

See https://gitlab.com/sc0ttj/dotfiles for lots of cool Puppy CLI stuff.

Terminal UIs (Reducing typing)

While we should always have a UNIX-style CLI available, we should remember some (many) users avoid the command line cos "it's too much typing" or "too much reading".

They don't want to type commands. We could/can use ncurses, but many say it's ugly or scary.

However...

We can also use our preferred "terminal picker", like fzf, picky, fff or similar, to make colourful, easy-to-use terminal menus for our CLI programs.

Some pickers:

Here's a list of liquid filters that can be used in the terminal:

https://github.com/sc0ttj/mdsh/blob/gh-pages/.app/functions/liquid_filters.bash

There are lots of "filters" ... Best used by sourcing it in a script to keep the rest of the code cleaner ... They aim to replace stuff like sed with much easier things:

echo "$something" | replace_first 'oldword' 'newword'

And provide new things:

echo 1000 | money £

Possible first step

Decide on the best, fastest, statically compiled terminal picker, and a small dialog statically compiled and put them into the initramfs, so every Puppy has ncurses and pick available right from boot..

Nobody probably knows but many small apps are cli/gui apps in a single script, may not be very nice to have cli/gui in the same script, but it's already there.

But the big apps certaintly aren't.

I have a working cli network manager derived from fatdog... well it was working 2 years ago. cli/gui .. I was about to integrate into woof 2 years ago, but the gui implementation was not finished and probably needed more fixes.

The truth is that you can't expect miracles, people do what they want at their own pace, the only way to make them work as a team is through money under contract. woof-CE is the living proof where basically none of the members know what the others are doing.

Your inputrc is already in woofce, you opened the PR, waited a few minutes, closed it and deleted your fork.

Nov 2018 https://github.com/puppylinux-woof-CE/woof-CE/blob/testing/woof-code/rootfs-skeleton/etc/inputrc

Thousands of improvements happened, but nobody noticed. Basically because puppy was so broken that fixing it a bit does take thousands of small fixes and improvements.

Some things are not possible with woof without a huge amount of effort, but they are possible with the debiandogs and so on, which is a subtle layer above debian.

Nobody probably knows but many small apps are cli/gui apps in a single script,

Everyone knows this... It's the problem I want to address.

I have a working cli network manager derived from fatdog... well it was working 2 years ago. cli/gui .. I was about to integrate into woof 2 years ago, but the gui implementation was not finished and probably needed more fixes.

I have a few different ncurses wifi tools I found.. some work, some need fixing up.. The Uis are fine, but I know nothing about wpa_supplicant..

Your inputrc is already in woofce, you opened the PR, waited a few minutes, closed it and deleted your fork.

That's nice... Didn't realise it got merged in, especially as I had to add it all in manually in Dpup Stretch to make it work..

Thousands of improvements happened, but nobody noticed.

Maybe if we had nicer docs - proper READMEs, nice commit messages, proper branches and PRs with good descriptive READMEs, separate repos for each important puppy program - then these changes wouldn't be so hard to find/see - there would be a lot more READMEs surfaced straight to the user as they browse the code/repo history.

The truth is that you can't expect miracles, people do what they want at their own pace

Every single time I make a suggestion I am met with this same response.. I am not expecting people to simply do my bidding, but trying to start a discussion about a better way forward...

Anyway... I will certainly create a split_cli_from_gui branch, and create a branch off that called split_cli_from_gui--advert-blocker, and I will make advert blocker separate CLI and GUI apps...

I will do a nice PR readme, with checkboxes and example/template code for others to use in their own .. Should get the ball rolling at least... The proof will be in the pudding.

I can create a new repo or a branch for my network manager, or maybe a single repo for everything in experimental state, something like staging. It's small enough.

My netwiz contains code from:

fatdog network setup
dougal's network wizard
simple network setup

The unfinished GUI is woofce-specific, the cli might be more generic. Providing stuff compatible with all puppy versions and universal stuff for everyone is not my goal.

It's meant to replace network wizard and SNS and become da built-in network manager. It's simple and advanced, but unfinished. It's one of the many unfinished projects I have.

rserwin1 might want to take look at the code, it's a bit old by now, but

I read that peasy wifi might contain the best code to deal with wifi

I think it's about reading and applying the best code no matter where it comes from, as long as it can be ported and applied successfully... integrating somebody else's code is a huge task.

Creating repos and commenting on git is as far I'll go. But it's almost Christmas, I may become quite busy anytime. 2020 will be a different year.

puppylinux-woof-CE / woof-CE