Here is a complete refactor of the Netkit-JH codebase. It mainly focuses on cleaning up the code, especially in removing common Bash pitfalls as identified by ShellCheck. Some functionality has been added or modified, and some features have been removed if deprecated or redundant; because of this the major version number will be incremented to 2.0.0-beta - though backwards compatibility largely still exists for regular usage.
NOTE: the update checker will now notify of both a new stable release and new prerelease. So version 2.0.0-beta will display alongside the notification for 1.1.4 if the user is on a version <1.1.4.
The man pages and Makefiles have not been updated yet.
The refactor has been tested manually to the best of my abilities, but understandably there may be bugs in less commonly used options or if the user is doing something very funky.
Notable features include:
Whitespace in arguments or paths does not completely break operation
netkit.conf.default stores default configuration variables that can be overridden with netkit.conf
Multiple tap interfaces can coexist and are user-specific (they are now specified with DOMAIN,TAP-ADDR,GUEST-ADDR, where DOMAIN doesn't have to be "tap")
There is now Bash tab completion
(For development) ShellCheck directives have been added and copyright notices now feature the Netkit-JH development team
The full changelog is as follows:
User Experience Changelog
Added
Command tab-completion (in Bash) including machine name and collision domain completion
--mount option to vstart - mount any host folder inside a machine
Ability to use a _test/shared.test script to run the same script for all hosts with ltest. Script output is stored in _test/results/$hostname.shared
Multiple taps can be created for machines to have their own unique interfaces (they can still share if the same collision domain is used)
Support for whitespace (and some other special characters) in arguments
Support for whitespace in the lab directory path
--clean-directories option to vclean to wipe $HOME/.netkit/ (ignoring netkit.conf), $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/ (now included in --clean-all)
Other Added
-v|--verbose option to lcrash, vclean, vconf, vcrash, and vdump
-f|--force option to vclean for usage alongside -H|--remove-hubs to remove hubs even if they are being used by running machines
--delay option to ltest to specify how long to wait for the lab to settle before running test scripts
--show-boot-log option to vstart to display the boot log (replaced the -v|--verbose option, see Other Modified)
--version option to install-netkit-jh.sh
Ability to affect a specific user for -T|--remove-tunnels in vclean
bash, binutils, coreutils, iproute2, lsof, and util-linux as dependencies
Modified
Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf
Tap interfaces are specified with DOMAIN,TAP-ADDR,GUEST-ADDR (DOMAIN does not have to be tap now, it should be a regular collision domain name). See Added for information on multiple tap interface specifications
vcommand usage signature is now vcommand [OPTION]... MACHINE [COMMAND]
vconnect usage signature is now vconnect [OPTION]... MACHINE
machines lab.conf variable is now LAB_MACHINES
Minimum Bash version (4.4) is enforced in the 1_check_shell.sh configuration check script
LAB_MACHINES variable is comma-delimited (orig. whitespace-delimited)
ltest user defined scripts should be stored in _test/scripts/
ltest script results are placed in _test/results/
'tap' interfaces are created under a /24 subnet now (not /8). This allows more 'tap' interfaces on the host to occupy private address spaces
Other Modified
vconf manual entry is called vconf now (used to be vconfig)
-v|--verbose in vstart now only displays machine information and ran commands. Boot log verbosity is now controlled with --show-boot-log, see Other Added
Rename --kill to --just-kill in vcrash and lcrash
-p|--print option has been changed to -n|--dry-run|--just-print|--recon for vconf, vpackage, and vstart. This follows what the Make utility uses
--clean-all in vclean now removes all items inside Netkit directories ($HOME/.netkit/, $MCONSOLE_DIR/, and $HUB_SOCKET_DIR/)
--clean-all in vclean now affects machines owned by all users
Machine names must conform to the Debian standard (which conforms to the RFC standard too). See hostname_regex in core/bin/script_utils for more guidance
Collision domain names must conform to a regular expression and length-check. See collision_domain_name_regex in core/bin/script_utils for more guidance
Better output when multiple labs are specified in linfo -m
tmux sessions are now named netkit-vhost (orig. netkit-vm)
Tap interfaces are named nk_$interface_sha256, where interface_sha256 is an 12-character (truncated) SHA-256 digest of the collision domain name SHA-256 appended to the owner's username's SHA-256 for uniqueness and avoiding character set issues
iptables forwarding rule uses the new interface specification to match the input interface (nk_+)
Virtual network hub sockets are named with the username's and domain's SHA-256 digests instead of plaintext to avoid issues with invalid characters. Tap sockets are identfied in the filename
/etc/vhostconfigured (file indicating machine has been booted before) is now at /etc/netkit/.vhostconfigured
Removed
--machine and --command switches in vcommand (see Modified)
ltest signature comparison, and the -R|--rebuild-signature and --verify options
-S|--script-mode option from ltest
USE_SUDO configuration directive (default is to use sudo now, su -c has been removed)
--quiet option for lcrash, lhalt, vclean, vcrash, and vhalt (-q|--quick still exists)
-q|--quiet option from vconf and vstart
konsole-tab argument to the --xterm option (and the ktabstart script)
bspwm compatibility patch files. They fail with the radically changed target files and fix a general issue other users have identified but only for bspwm - it applies to tmux, Windows Terminal, and other terminal emulators and window managers
Dependency on the net-tools (ifconfig) package for manage_tuntap in favour of iproute2
Other Removed
Port helper compatibility
-w|--hostwd option from vstart (it did not provide any purpose)
lrestart symbolic link (pointed to lstart and provided no default or additional functionality)
--fix option from check_configuration.sh
MCONSOLE_DIR environment variable detection from uml_mconsole
Support for netkit.conf overriding with the NETKIT_FILESYSTEM, NETKIT_MEMORY, NETKIT_KERNEL, and NETKIT_TERM environment variables
01-check_path.sh setup script (whitespace in paths is safely handled now)
06-check_aux_tools.sh and 10-check_dumpers.sh setup scripts
Development Changelog
Added
Netkit's default configuration is stored in netkit.conf.default. This should not be modified and can be overridden with netkit.conf. This allows for smoother updates of configuration variables and their default values, since the default file can be entirely replaced with the new version (as opposed to using a janky sed script)
ShellCheck directives for effective linting
.gitignore now ignores compilation, test, and development artefacts
Copyright notices for the Netkit-JH development team (alongside the original Netkit and Netkit-NG notices)
Collision domain argument to manage_tuntap
vcommon source script to contain functions common to the vcommands (analogous to lcommon)
Modifed
All scripts are interpreted by Bash now (minimum required version 4.4, see Modified)
Script order:
/usr/bin/env bash shebang
Copyright notice
Script description
usage_line & usage
Other function definitions in order of usage
script_utils and (optional) lcommon/vcommon source
getopt argument parsing
Non-option argument parsing
Remainder of script
hostname parameter in kernel_cmd (replaced name and title)
ltest automated lab testing now works (added /etc/netkit/netkit-test-phase)
Known Issues
The man pages have not been updated yet
Removing an interface with vconf crashes the machine
Whitespace in the VM_MODEL_FS (-m|--model-fs) and VM_KERNEL (--kernel) configuration variables, HOME environment variable, or the argument to --mount, --hostlab, and --exec may break when ran with Windows Terminal due to escaped quotes not being honoured by the extra layer of shell interpretation.
A terminating quotation mark in the VM_MODEL_FS (-m|--model-fs) configuration variable will be removed by the get_vhost_info_by_pid function if the instance is running in Windows Terminal. This is significant in vcrash's and vhalt's --remove-fs option - the disk file will not be removed (and a file with the same name sans quote mark will be, if exists).
Here is a complete refactor of the Netkit-JH codebase. It mainly focuses on cleaning up the code, especially in removing common Bash pitfalls as identified by ShellCheck. Some functionality has been added or modified, and some features have been removed if deprecated or redundant; because of this the major version number will be incremented to
2.0.0-beta- though backwards compatibility largely still exists for regular usage.NOTE: the update checker will now notify of both a new stable release and new prerelease. So version 2.0.0-beta will display alongside the notification for 1.1.4 if the user is on a version <1.1.4.
The man pages and Makefiles have not been updated yet.
The refactor has been tested manually to the best of my abilities, but understandably there may be bugs in less commonly used options or if the user is doing something very funky.
Notable features include:
DOMAIN,TAP-ADDR,GUEST-ADDR, whereDOMAINdoesn't have to be "tap")The full changelog is as follows:
User Experience Changelog
Added
--mountoption tovstart- mount any host folder inside a machineltest. Script output is stored in _test/results/$hostname.shared--clean-directoriesoption tovcleanto wipe$HOME/.netkit/ (ignoring netkit.conf),$MCONSOLE_DIR/, and$HUB_SOCKET_DIR/ (now included in--clean-all)Other Added
-v|--verboseoption tolcrash,vclean,vconf,vcrash, andvdump-f|--forceoption tovcleanfor usage alongside-H|--remove-hubsto remove hubs even if they are being used by running machines--delayoption toltestto specify how long to wait for the lab to settle before running test scripts--show-boot-logoption tovstartto display the boot log (replaced the-v|--verboseoption, see Other Modified)--versionoption to install-netkit-jh.sh-T|--remove-tunnelsinvcleanbash,binutils,coreutils,iproute2,lsof, andutil-linuxas dependenciesModified
DOMAIN,TAP-ADDR,GUEST-ADDR(DOMAINdoes not have to betapnow, it should be a regular collision domain name). See Added for information on multiple tap interface specificationsvcommandusage signature is nowvcommand [OPTION]... MACHINE [COMMAND]vconnectusage signature is nowvconnect [OPTION]... MACHINEmachineslab.conf variable is nowLAB_MACHINESLAB_MACHINESvariable is comma-delimited (orig. whitespace-delimited)ltestuser defined scripts should be stored in _test/scripts/ltestscript results are placed in _test/results/Other Modified
vconfmanual entry is calledvconfnow (used to bevconfig)-v|--verboseinvstartnow only displays machine information and ran commands. Boot log verbosity is now controlled with--show-boot-log, see Other Added--killto--just-killinvcrashandlcrash-p|--printoption has been changed to-n|--dry-run|--just-print|--reconforvconf,vpackage, andvstart. This follows what the Make utility uses--clean-allinvcleannow removes all items inside Netkit directories ($HOME/.netkit/,$MCONSOLE_DIR/, and$HUB_SOCKET_DIR/)--clean-allinvcleannow affects machines owned by all usershostname_regexin core/bin/script_utils for more guidancecollision_domain_name_regexin core/bin/script_utils for more guidancelinfo -mnetkit-vhost(orig.netkit-vm)nk_$interface_sha256, whereinterface_sha256is an 12-character (truncated) SHA-256 digest of the collision domain name SHA-256 appended to the owner's username's SHA-256 for uniqueness and avoiding character set issuesiptablesforwarding rule uses the new interface specification to match the input interface (nk_+)Removed
--machineand--commandswitches invcommand(see Modified)--machineswitch invconnect(see Modified)ltestsignature comparison, and the-R|--rebuild-signatureand--verifyoptions-S|--script-modeoption fromltestUSE_SUDOconfiguration directive (default is to usesudonow,su -chas been removed)--quietoption forlcrash,lhalt,vclean,vcrash, andvhalt(-q|--quickstill exists)-q|--quietoption fromvconfandvstartkonsole-tabargument to the--xtermoption (and the ktabstart script)net-tools(ifconfig) package formanage_tuntapin favour ofiproute2Other Removed
-w|--hostwdoption fromvstart(it did not provide any purpose)lrestartsymbolic link (pointed tolstartand provided no default or additional functionality)--fixoption from check_configuration.shMCONSOLE_DIRenvironment variable detection from uml_mconsoleNETKIT_FILESYSTEM,NETKIT_MEMORY,NETKIT_KERNEL, andNETKIT_TERMenvironment variablesDevelopment Changelog
Added
sedscript)manage_tuntapModifed
/usr/bin/env bashshebangusage_line&usagescript_utilsand (optional)lcommon/vcommonsourcegetoptargument parsinghostnameparameter inkernel_cmd(replacednameandtitle)Removed
nameandtitleparameters inkernel_cmdBug Fixes
uml_switchlogging now worksltestautomated lab testing now works (added /etc/netkit/netkit-test-phase)Known Issues
vconfcrashes the machineVM_MODEL_FS(-m|--model-fs) andVM_KERNEL(--kernel) configuration variables,HOMEenvironment variable, or the argument to--mount,--hostlab, and--execmay break when ran with Windows Terminal due to escaped quotes not being honoured by the extra layer of shell interpretation.VM_MODEL_FS(-m|--model-fs) configuration variable will be removed by theget_vhost_info_by_pidfunction if the instance is running in Windows Terminal. This is significant invcrash's andvhalt's--remove-fsoption - the disk file will not be removed (and a file with the same name sans quote mark will be, if exists).