search

Pack3tL0ss / ConsolePi

Raspberry Pi Based Serial Console Server, with PushBullet Notification of IP changes, Automatic VPN termination, custom menu, Power Outlet Control, and a lot more
MIT License
178 stars 15 forks source link
bash hostapd hotspot hotspot-mode openvpn pushbullet pushbullet-api python raspberry-pi raspberrypi ser2net serial-adapters serial-ports telnet udev-rules wpa-supplicant

readme

ConsolePi

Acts as a serial Console Server, allowing you to remotely connect to ConsolePi via Telnet/SSH/bluetooth to gain Console Access to devices connected to local or remote ConsolePis via USB to serial adapters (i.e. Switches, Routers, Access Points... anything with a serial port).

TL;DR: Single Command Install Script. Run from an internet connected RaspberryPi running RaspiOS (ideally a fresh image). See Known Issues regarding changes in Raspberry Pi OS 12(bookworm).

wget -q https://raw.githubusercontent.com/Pack3tL0ss/ConsolePi/master/installer/install.sh -O /tmp/ConsolePi && sudo bash /tmp/ConsolePi && sudo rm -f /tmp/ConsolePi

consolepi animated Demo

Making multiple ConsolePis? Want to re-image, but pull-over all of your existing configs? Check out the image creator!!!

Contents

Known Issues

What's New

Feb 2024 (v2024-3.0 installer v80)

✨ Large update!!

The release of Raspberry Pi OS 12 (bookworm) included a change to use NetworkManager to manage the network. That broke all network based automations (PushBullet notifications of IP change, cloud sync after IP change, Automatic VPN, Auto fallback to hotspot, and ZTP (fallback to static wired w/ DHCP))

Here is a summary of what's in this release:

Prior Changes can be found in the - ChangeLog

Jan 2024 (v2024-1.0)

July 2023 (v2023-6.1)

July 2023 (v2023-6.0)

Planned enhancements

Features

Feature Summary Image

consolepi-menu image

Serial Console Server

This is the core feature of ConsolePi.

Guidance on LAME USB to RS232 adapters

There are some lame adapters that don't burn a serial # to the chip, this makes assigning a unique name/TELNET port more challenging. Refer to the adapters page for more guidance.

Spoiler alert, Adapters based on an FTDI chipset FTW!

AutoHotSpot

This is handled by NetworkManager, but the installer deploys the appropriate config (if you've chosen to enable the feature). On boot NetworkManager will first look for any configured SSIDs (you need to build those profiles), if it can't find or is unable to connect it will fallback to hotspot. Any profiles you setup should have an autoconnect-priority=N where N is anything > 0. Consolpi will also configure DHCP for hotspot clients such that if the ConsolePi has no network reachability (it is isolated) it will not distribute a gateway to hotspot clients. This is specifically done so you can connect to a ConsolePi that is not network connected (to access connected serial ports) from a PC via a second NIC. This way it won't compete with your existing default-route (which would black-hole all your traffic).

If a domain is provided to the wired port via DHCP, and the hotspot is enabled ConsolePi will distribute that same domain via DHCP to clients.

On systems initially installed prior to Raspberry Pi OS 12 (bookworm) (Uses dhcpcd vs NetworkManager), autohotspot is managed by the autohotspot service which runs a script on boot, but functionality is the same.

Automatic VPN

When an interface receives an IP address ConsolePi will Automatically connect to a VPN server under the following conditions:

You need to configure the NetworkManager connection profile for your VPN. It is best to manually test it using nmcli con up <connection id> to verify the configuration. You may need to include the --ask option when you manually test it so NetworkManager will store any secrets involved. Refer to NetworkManager documentation for more detail.

Automatic PushBullet Notification

(Requires a PushBullet Account, API key, and the app / browser extension.)

When ConsolePi receives a dynamic IP address. A message is sent via PushBullet API with the IP so you know how to reach ConsolePi.

Push Bullet Notification image

An additional message is sent once a tunnel is established if the Automatic VPN feature is enabled.

Push Bullet Notification image

Each Time a Notification is triggered all interface IPs are sent in the message along with the ConsolePi's default gateway(s).

Automatic Wired DHCP Fallback

Use with caution

The Wired DHCP Fallback function configures the wired interface of ConsolePi to fallback to static when it fails to get an address as a client via DHCP. It will then enable a DHCP Server on the wired interface (with the ConsolePi as the gateway for the clients).

This is useful when configuring factory-default devices, or on an isolated staging network. Caution should be taken on production networks.

This function also:

The ZTP Orchestration feature will enable wired fallback to static/DHCP Server when you run consolepi-ztp. consolepi-ztp -end restores everything to pre-ZTP state. You do not need to enable it if ZTP is your only need for it consolepi-ztp will handle that.

ConsolePi Cluster / Cloud Sync

The Cluster feature allows you to have multiple ConsolePis connected to the network, or to each other (i.e. first ConsolePi in hotspot mode, the others connected as clients to that hotspot). A connection to any one of the ConsolePis in the Cluster will provide options to connect to any local serial adapters, as well as those connected to the other ConsolePis in the cluster (via the consolepi-menu command).

consolepi-menu image

Another look at the menu

consolepi-menu image

In this example only 1 adapter is connected locally (menu item 1), Items 2 - 26 are adapters discovered on other ConsolePis on the network. Items 27 - 50 are Manual Host Entries.

Supported Cluster Sync Methods:

Google Drive:

Read The Google Drive Setup for instructions on setting up Google Drive and authorizing ConsolePi to leverage the API.

  • Google Drive/Google Sheets is currently the only external method supported. Given this gets the job done, it unlikely more external methods will be added.

mDNS / API

Local Cloud Cache

Power Control

The Power Control Function allows you to control power to external outlets. ConsolePi supports:

The Power Control Menu

Defined Outlets will show up in the Power Menu

consolepi-menu image

Additional Controls for Digital Loggers

You may have an 8 port digital-loggers web power switch, with only some of those ports linked to devices. Any Outlets linked to devices will show up in the Power Menu, All of the Ports for the DLI (regardless of linkage) will show up in the dli menu

consolepi-menu image

Outlet Linkages

Example Outlet Linkage. In this case the switch "rw-6200T-sw" has 2 ports linked. Both are on a dli web power switch. One of the ports is for this switch, the other is for the up-stream switch that serves this rack. When connecting to the switch, ConsolePi will ensure the linked outlets are powered ON. ConsolePi does not power-off the outlets when you disconnect.


------------------------------------------------------------------------------------
  Ensuring r2-6200T-sw Linked Outlets (labpower2:[1, 4]) are Powered ON
------------------------------------------------------------------------------------

Connecting To r2-6200T-sw...
picocom v3.1
...

Refer to Power Control Setup for details on how to setup Power Control.

Manual Host Entries

The Manual Host Entries Feature allows you to manually define other SSH or TELNET endpoints that you want to appear in the menu. These entries will appear in the rs (remote shell) menu by default, but can also show-up in the main menu if show_in_main: true. Manual host entries support outlet bindings (Auto Power On when connecting through the menu).

Refer to Manual Host Entries in the Configuration section foe details on how to configure.

ZTP Orchestration

ConsolePi supports Zero Touch Provisioning(ZTP) of devices via wired ethernet/DHCP. The feature uses DHCP to trigger ZTP, and supports config file generation using jinja2 templates. For more details see ConsolePi ZTP Orchestration.

ConsolePi API

ConsolePi includes an API with the following available methods

All Are GET methods the default port is 5000, which can be overridden in the config via the api_port key in the optional OVERRIDES stanza. See ConsolePi.yaml.example.

/api/v1.0/

The swagger interface is @ /api/docs or /api/redoc. You can browse/try the less common API methods there.

The API is used by ConsolePi to verify reachability and ensure adapter data is current on menu-load.

The API is currently unsecured, it uses http, and Auth is not implemented yet. It currently only supports GET requests and doesn't provide any sensitive (credential) data.

ConsolePi Extras

Toward the end of the install, and via consolepi-extras anytime after the install, you are provided with options to automate the deployment (and removal for most) of some additional tools. This is a selection of tools not required for ConsolePi, but often desired, or useful for the kind of folks that would be using ConsolePi.

`consolepi-extras`

Note: speed test (locally hosted browser based speed-test (server)), is only presented as an option for Pi4/CM4+. Devices prior had a slow ethernet port, so it wouldn't make sense to use them as a speed-test server.

Installation

If you have a Linux system available you can use ConsolePi image creator to burn the image to a micro-sd, enable SSH, mass-import configurations (if ran from a ConsolePi, optional), and PreConfigure ConsolePi settings (optional). This script is especially useful for doing headless installations.

The Following Applies to All Automated Installation methods

Note Previous versions of ConsolePi supported import from either the users home-dir (i.e. /home/wade) or from a consolepi-stage subdir in the users home-dir (i.e. /home/wade/ConsolePi-stage). The import logic directly from the home-dir has not been removed, but going forward any new imports will only be tested using the consolePi-stage directory for simplicity.

ConsolePi will optionally use pre-configured settings for the following if they are placed in the a consolepi-stage subdir in the users home folder (i.e. /home/wade/consolepi-stage). This is optional, the installer will prompt for the information if not pre-configured. It will prompt you to verify either way. Imports only occur during initial install not upgrades.

staged-file import also supports host specific folders within consolepi-stage. The installer will first check if there is a folder that matches the CosnolePis hostname (you can set via the --hostname flag when you run the installer, if it's still the default.). So if i'm installing on a system already set with the hostname ConsolePi5 or i've launched the installer with the --hostname ConsolePi5 option. Any imports described below found in consolepi-stage/ConsolePi5 will take precedence over anything found in the root of consolepi-stage

Examples below use consolepi-stage/... for brevity but consolepi-stage/HOSTNAME/... is valid for all examples

To enable the Clustering / Cloud-Sync function see the description above and the prerequisite Google Drive Setup instructions.

1. Automated Installation

Install RaspberryPi OS on a raspberryPi and connect it to the network.

Use the command string below to kick-off the automated installer. The install script is designed to be essentially turn-key. It will prompt to change hostname, set timezone, and set the consolepi users password. Be sure to checkout the image creator script if doing a headless install, creating multiple ConsolePis, or want to re-image an existing ConsolePi.

wget -q https://raw.githubusercontent.com/Pack3tL0ss/ConsolePi/master/installer/install.sh -O /tmp/ConsolePi && sudo bash /tmp/ConsolePi && sudo rm -f /tmp/ConsolePi

Command String Breakdown:

wget -q https://raw.githubusercontent.com/Pack3tL0ss/ConsolePi/master/installer/install.sh -O /tmp/ConsolePi --> Pull current install script from GitHub and place in /tmp/ConsolePi

sudo bash /tmp/ConsolePi --> Execute ConsolePi install script with sudo privs

sudo rm -f /tmp/ConsolePi --> Remove the install script (script execution is complete at this point)

The && between each command just means the command will execute given there was not an error returned from the previous command.

Silent Install

A Silent install (Installation runs without prompts) is possible cmd line arguments provided to the installer or a config file, where the path to the config is provided to the installer via the -C </path/to/config/file.conf> argument. A pre configured ConsolePi.yaml should also exist in the consolepi-stage dir described above.

Refer to /etc/ConsolePi/installer/install.conf.example for an example config. This command string will download it to your home dir as install.conf and open it in nano for editing.

wget -q https://raw.githubusercontent.com/Pack3tL0ss/ConsolePi/master/installer/install.conf.example -O ~/install.conf && nano ~/install.conf

The output below shows consolepi-upgrade as the command to launch, the command will be consolepi-install on an image created using the image creator script. If neither is the case you would call the installer directly (with sudo) and pass in the args (The TL;DR string at the top of this README can be modified to pass in the arguments)

wade@ConsolePi-dev:~$ consolepi-upgrade --help

USAGE: consolepi-upgrade [OPTIONS]

Available Options
 -h|--help                               Display this help text.
 -s|--silent                             Perform silent install no prompts, all variables reqd must be provided via pre-staged configs.
 -C | --config <path/to/config>          Specify config file to import for install variables (see /etc/ConsolePi/installer/install.conf.example).
    Copy the example file to your home dir and make edits to use.
 -6|--no-ipv6                            Disable IPv6 (Only applies to initial install).
 -R|--reboot                             reboot automatically after silent install (Only applies to silent install).
 -w|--wlan_country <2 char country code> wlan regulatory domain (Default: US).
 --locale <2 char country code>          Update locale and keyboard layout. i.e. '--locale us' will result in locale being updated to en_US.UTF-8.
 --us                                    Alternative to the 2 above, implies us for wlan country, locale, and keyboard..
 -H|--hostname <hostname>                If set will bypass prompt for hostname and set based on this value (during initial install).
 --tz <tz>                               Configure TimeZone i.e. America/Chicago.
 -L|--auto-launch                        Automatically launch consolepi-menu for consolepi user.  Defaults to False..
 -p|--passwd '<password>'                Use single quotes: The password to configure for consolepi user during install..
    Any manually added users should be members of 'dialout' and 'consolepi' groups for ConsolePi to function properly

The Following optional arguments are more for dev, but can be useful in some other scenarios
 -P|--post                               ~/consolepi-stage/consolepi-post.sh if found is executed after initial install.  Use this to run after upgrade..
 --no-apt                                Skip the apt update/upgrade portion of the Upgrade.  Should not be used on initial installs..
 --no-pip                                Skip pip install -r requirements.txt.  Should not be used on initial installs..

Examples:
  This example specifies a config file with -C (telling it to get some info from the specified config) as well as the silent install option (no prompts)
        > consolepi-upgrade -C /home/consolepi/consolepi-stage/installer.conf --silent

  Alternatively the necessary arguments can be passed in via cmd line arguments
  NOTE: Showing minimum required options for a silent install.  ConsolePi.yaml has to exist
        wlan_country will default to US, No changes will be made re timezone, ipv6 & hostname
        > consolepi-upgrade -silent -p 'c0nS0lePi!'

2. Semi-Automatic Install

Alternatively you can clone this repository to /etc manually, then run the install script. The only real benefit here is it would allow you to cp ConsolePi.yaml.example ConsolePi.yaml and make edits to your liking. If ConsolePi.yaml exists when the installer runs you'll skip a number of user input steps and go straight to verification of the provided settings.

You still get the option when using the Automated Installation to stop the install after a default ConsolePi.yaml is created, allowing you to edit then re-run bypassing input prompts.

cd /etc
sudo git clone https://github.com/Pack3tL0ss/ConsolePi.git

- or -

cd /tmp
git clone https://github.com/Pack3tL0ss/ConsolePi.git
sudo mv /tmp/ConsolePi /etc

Optionally Pre-Configure parameters, it will result in less time on data-collection/user-input during the install. Just grab the ConsolePi.yaml.example file from the repo, edit it with your settings, and rename/place in ~/consolepi-stage/ConsolePi.yaml

# example assuming logged in as pi
cd ~
mkdir consolepi-stage
sudo cp ConsolePi.yaml.example ~/consolepi-stage/ConsolePi.yaml
sudo nano ConsolePi.yaml

This example copies the configuration to the stage directory to highlight that function (importing settings from the stage directory), you could also place the configured ConsolePi.yaml file in /etc/ConsolePi.

NOTE: pre-staging only occurs on the initial install, not when using consolepi-upgrade.

Configure parameters to your liking then

sudo /etc/ConsolePi/installer/install.sh

3. ConsolePi Image Creator

!!WARNING!! This script writes RaspberryPi OS to a connected micro-sd card. This will overwrite everything on that card. If something doesn't look right STOP. With that said I've used it 100s of times by now, so image away.

From an Existing ConsolePi:

OR

Using a Linux System (Most distros should work only requirement is a bash shell ... tested on RaspiOS and Mint) enter the following command:

ConsolePi_image_creator brief summary:

The "Stage dir" referenced below is a sub directory found in cwd (your current working directory). The script looks for the Stage dir which needs to be named 'consolepi-stage' and moves the entire directory to the pi users home directory on the media being imaged. When the installer runs (on the new ConsolePi) it will automatically import config items from that staging directory.

The Pre-staging described below is optional, this script can be used without any pre-staging files, it will simply burn a RaspiOS image to the micro-sd, enable SSH, and set the installer to run automatically on boot (unless you set auto_install to false via cmd line arg or config).

USAGE: consolepi-image [OPTIONS]

Available Options
 -h|--help                               Display this help text.
 -C <location of config file>            Look @ Specified config file loc to get command line values vs. the default consolepi-image-creator.conf (in cwd).
 --ssid <ssid>                           Configure SSID on image (configure wpa_supplicant.conf).
 --psk '<psk>'                           Use single quotes: psk for SSID (must be provided if ssid is provided).
 --wlan-country <2 char country code>    wlan regulatory domain (Default: US).
 --priority <priority>                   wlan priority if specifying psk SSID via --ssid and --psk flags (Default 0).
 --img-type <lite|desktop|full>          Type of RaspiOS image to write to media (Default: lite).
 --img-only                              Only install RaspiOS, no pre-staging will be done other than enabling SSH (Default: false).
 --[no-]auto-install                     image will not be configured to auto launch the installer on first login (Default true).
 --[no-]import                           whether or not to import files from this system to the image, if this is a ConsolePi.  Prompted if not set..
 --[no-]edit                             Skips prompt asking if you want to edit (nano) the imported ConsolePi.yaml..
 -H|--hostname                           pre-configure hostname on image..
 -I|--image                              Use specified image (full path or file in cwd).
 -p|--passwd <consolepi password>        The password to set for the consolepi user..
 --cmd-line '<cmd_line arguments>'       *Use single quotes* cmd line arguments passed on to 'consolepi-install' cmd/script on image.

The consolepi-image-creator will also look for consolepi-image-creator.conf in the current working directory for the above settings

Examples:
  This example overrides the default RaspiOS image type (lite) in favor of the desktop image and configures a psk SSID (use single quotes if special characters exist)
        sudo ./consolepi-image-creator.sh --img-type desktop --ssid MySSID --psk 'ConsolePi!!!'
  This example passes the -C option to the installer (telling it to get some info from the specified config) as well as the silent install option (no prompts)
        sudo ./consolepi-image-creator.sh --cmd-line='-C /home/consolepi/consolepi-stage/installer.conf --silent'
# ----------------------------------- // DEFAULTS \\ -----------------------------------
# ssid: No Default ~ psk ssid not configured if ssid and psk is not provided
# psk: No Default
# wlan_country: "us"
# priority: 0
# img_type: "lite"
# img_only: false
# auto_install: true
# mass_import: Not Set (Will Prompt User)
#    mass_import=true will bypass the prompt and do the import
#    mass_import=false will bypass the prompt and will not perform the import
# edit: Not Set (Will Prompt User)
#    edit=true will bypass the prompt and open the staged ConsolePi.yaml for editing
#    edit=false will bypass, the prompt ConsolePi.yaml will remain as imported
# hotspot_hostname: Not Set (Will Prompt User)
#    edit=true will pre-configure the hostname on the image to match the HotSpot SSID
#    edit=false will bypass prompt and leave hostname as default (raspberrypi)
# --------------------------------------------------------------------------------------

What the script does

This capture highlights what the script does and what it pulls via mass import if ran from an existing ConsolePi

pi@ConsolePi-dev:~$ sudo ./consolepi-image-creator.sh
   ______                       __     ____  _
  / ____/___  ____  _________  / /__  / __ \(_)
 / /   / __ \/ __ \/ ___/ __ \/ / _ \/ /_/ / /
/ /___/ /_/ / / / (__  ) /_/ / /  __/ ____/ /
\____/\____/_/ /_/____/\____/_/\___/_/   /_/
  https://github.com/Pack3tL0ss/ConsolePi

ConsolePi Image Creator
'exit' (which will terminate the script) is valid at all prompts

Script has discovered removable flash device @ sda with the following details

------------------------------- // Device Details for sda \\ -----------------------------------

Disk /dev/sda: 29.8 GiB, 32010928128 bytes, 62521344 sectors
Disk model: Storage Device
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe34e0aca

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sda1         8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/sda2       532480 62521343 61988864 29.6G 83 Linux

------------------------------------------------------------------------------------------------

Press enter to accept sda as the destination drive or specify the correct device (i.e. 'sdc' or 'mmcblk0')
Device to flash with image [sda]:

Getting latest raspios image (lite)
Using image 2023-05-03-raspios-bullseye-armhf-lite found in /home/wade. It is the current release

!!! Last chance to abort !!!
About to write image 2023-05-03-raspios-bullseye-armhf-lite.img to sda, Continue? (y/n|exit): y
   ______                       __     ____  _
  / ____/___  ____  _________  / /__  / __ \(_)
 / /   / __ \/ __ \/ ___/ __ \/ / _ \/ /_/ / /
/ /___/ /_/ / / / (__  ) /_/ / /  __/ ____/ /
\____/\____/_/ /_/____/\____/_/\___/_/   /_/
  https://github.com/Pack3tL0ss/ConsolePi

Now Writing image 2023-05-03-raspios-bullseye-armhf-lite.img to sda standby...
 This takes a few minutes

1849688064 bytes (1.8 GB, 1.7 GiB) copied, 221 s, 8.4 MB/s
442+0 records in
442+0 records out
1853882368 bytes (1.9 GB, 1.7 GiB) copied, 233.167 s, 8.0 MB/s

Image written to flash - no Errors

 ~ Mounting boot partition to enable ssh.................................OK
 ~ Enabling ssh on image.................................................OK
 ~ unmount boot partition................................................OK
 ~ Mounting System partition to pre-configure ConsolePi image............OK
 ~ Script Option to pre-config psk ssid..................................Skipped - Not Configured
 ~ Configure Auto-Install on first login.................................OK
     Configured with the following args -C consolepi-stage/install.conf
 ~ consolepi-stage dir found Pre-Staging all files.......................OK
 ~ SSH authorized keys found pre-staging.................................OK
 ~ SSH known_hosts found pre-staging.....................................OK
 ~ wpa_supplicant.conf found pre-staging on image........................OK
 ~ staged wpa_supplicant includes EAP-TLS SSID looking for certs.........OK

   -- Performing Imports from This ConsolePi --
 ~ /etc/ConsolePi/ConsolePi.yaml.........................................Skipped - Already Staged
 ~ /etc/wpa_supplicant/wpa_supplicant.conf...............................Skipped - Already Staged
 ~ /etc/udev/rules.d/10-ConsolePi.rules..................................Skipped - Already Staged
 ~ /etc/ser2net.conf.....................................................Skipped - Already Staged
 ~ /home/wade/.ssh/authorized_keys.......................................Skipped - Already Staged
 ~ /home/wade/.ssh/known_hosts...........................................Skipped - Already Staged
 ~ /etc/ConsolePi/cloud/gdrive/.credentials/credentials.json.............Imported
 ~ /etc/ConsolePi/cloud/gdrive/.credentials/token.pickle.................Imported
 ~ /etc/openvpn/client/ConsolePi.ovpn....................................Skipped - Already Staged
 ~ /etc/openvpn/client/ovpn_credentials..................................Skipped - Already Staged

Consolepi image ready

Once Complete you place the newly blessed micro-sd in your raspberryPi and boot. The installer will automatically start unless you've disabled it. In which case the consolepi-install will launch the installer (unless img_only=true, if so consolepi-install command is not created).

Alternative Hardware Installs

For my use case I manually installed these, but depending on the system/use-case you could use the installer in the same method described above. results may vary :)

Testing and extending the installer to natively detect and support installation on non RaspberryPi devices is planned for a future release. It currently has not been tested.

The Use Cases

  1. ConsolePi running on a Linux Mint laptop

    • desire was to be able to load the menu, see the remotes, and use a locally connected adapter if I wanted, but to only sync one way (discover remote ConsolePis, but don't advertise to them). This is because the laptop is used ad-hoc and if I'm using it I'm on it, not remote.

    • Install Process was simply (this is from memory so might be off a bit):

      sudo apt install python3-pip virtualenv git
cd /tmp
git clone https://github.com/Pack3tL0ss/ConsolePi.git
cd ConsolePi  # Should now be in /tmp/ConsolePi
python3 -m virtualenv venv
sudo mv /tmp/ConsolePi /etc
sudo cp /etc/ConsolePi/src/consolepi.sh /etc/profile.d && . /etc/profile.d/consolepi.sh # <-- adds consolepi-commands to PATH
consolepi-sync -pip # <-- updates perms installs (pip) requirements
cp /etc/ConsolePi/ConsolePi.yaml.example /etc/ConsolePi/ConsolePi.yaml
consolepi-config # <-- edit as required ~ `cloud_pull_only: true` option in the OVERRIDES: section for this use case.
sudo cp /etc/ConsolePi/src/systemd/consolepi-mdnsbrowse.service /etc/systemd/system
sudo systemctl enable consolepi-mdnsbrowse
sudo systemctl start consolepi-mdnsbrowse
consolepi-menu # <-- See Note Below Regarding initial cloud AuthZ if using Gdrive Sync

      test the menu (note if cloud sync enabled you still need to put the creds in the dir). Select option r (refresh) if cloud enabled and creds in place. If you've completed Google Drive Setup, and need to authorize ConsolePi for the first time launch the menu with the cloud argument (consolepi-menu cloud) (then select the r (refresh) option). This is only required to create the credential files for the first time, you can use consolepi-menu without arguments to launch after the creds have been created.

      I think that's it. So the above will allow use of the menu on the LapTop, will detect any local adapters if any are plugged in, will discover and allow connection to any remotes, manually defined hosts, power outlets, etc, but will not advertise itself to any other ConsolePis.

      If you did want the system to advertise itself on the network, so other ConsolePis could discover it: Repeat the commands above related to consolepi-mdnsbrowse.service but swap in consolepi-mdnsreg.service.

  2. ConsolePi running on wsl-ubuntu (Windows Subsystem for Linux)

    • Use Case... I always have a wsl terminal open, so I use the consolepi-menu in wsl, which displays all the discovered remote consolepis.
    • No local-adapters wsl would be remote only.
    • Install process: Same as above with the exception of leave out the consolpi-mdnsbrowse bit (no systemd on wsl).
    • It works as expected, with the minor caveat that it's only source to get remote details is via cloud-sync. Adapter data is still refreshed on menu-load by querying the remote directly. You also can not create the cloud credentials files (do the initial Authorization) in wsl. That needs to be done on another system and copied over.

    For Alternative installs. Use consolepi-sync to update ConsolePi rather than consolepi-upgrade. or just git pull from the /etc/ConsolePi directory. consolepi-upgrade is the installer (which will detect ConsolePi is already installed and run as upgrade), has not been tested on non RaspberryPi installs yet.

ConsolePi Usage

Configuration

The Configuration file is validated and created during the install. Settings can be modified post-install via the configuration file /etc/ConsolePi.yaml (Some Changes will require consolepi-upgrade to be ran to take effect). See ConsolePi.yaml.example for an example of the available options.

consolepi-menu sorting and connection settings

When you assign a friendly alias to an adapter for predictability via the rn (rename) option in consolepi-menu or via consolepi-addconsole an alias (udev rule) is created for that adapter and ser2net.conf is updated with a pointer to that alias using the next available TELNET port in the 7xxx range which includes the desired serial settings. The consolepi-menu parses the ser2net.conf to retrieve the serial settings for each device, but it also uses this file to determine the order the adapters appear in the menu. The menu is sorted by TELNET port#. So if you want re-arrange the order devices show up you just need to re-arrange the port #s mapped in /etc/ser2net.conf for the devices. Just ensure each device is mapped to a unique port (no duplicate ports).

You can use the tp option in consolepi-menu to display the TELNET ports mapped to each device. Re-arranging them still needs to be done manually by editing /etc/ser2net.conf

Configuring Manual Host Entries

The Manual Host Entries Feature allows you to manually define other SSH or TELNET endpoints that you want to appear in the menu. These entries will appear in the rs remote shell menu by default, but can also show-up in the main menu if show_in_main: true. Manual host entries support power outlet bindings as well. To configure this feature simply populate the optional HOSTS: section of ConsolePi.yaml. Using the following structure:

HOSTS:
  mm1(serial):
    address: 10.0.30.60:7001
    method: telnet  # This field is now optional defaults to ssh if not specified.
    show_in_main: true
    group: WLAN
  mc1(ssh):
    address: 10.0.30.24
    method: ssh
    username: wade
    show_in_main: true
    key: wade_arubaos_pub.key
    group: WLAN
  LabDigi1:
    address: labdigi.kabrew.com
    method: ssh
    username: wade
  omv:
    address: omv.kabrew.com
    method: ssh
    username: root
    group: WADELAB-HOSTS

The Above Example highlights different options

Local UART Support

ConsolePi supports use of the onboard UARTs for external connections. The Pi4 actually has 6 UARTs onboard (5 useable). The additional UARTs would need to be enabled. The examples below should get you there if you want to make use of the extra UARTs, obviously you can search the internet or refer to the Pi4 data-sheet for info beyond that.

Note: The RaspberryPis onboard UARTs are TTL level. This is useful for connecting to other devices with TTL level UARTs (i.e. Another Rpi, Arduino, or Aruba APs that used the flat 4 pin connector (The grey Aruba Adapter used to connect to these APs AP-SER has a TTL to RS232 level shifter built into the cable)). To use these to connect to RS232 ports typically found on Network Hardware and other equipment you need a ttl<-->RS232 level shifter i.e. (max232 family).

To Enable Support for GPIO UARTs (ttyAMA):

Update 10-ConsolePi.rules if necessary

If /etc/udev/rules.d/10-ConsolePi.rules already exists on the system, and was created prior to v2020-4.5 (Sept 2020). You will likely need to update the file to support the additional GPIO UARTs on the Pi4. The GPIO UARTs were initially configured to use it's own separate rules file (11-ConsolePi-ttyama.rules). With v2020-4.5 it was added to the common rules file 10-CosnolePi.rules. The format/template used to build 10-ConsolePi.rules had to be updated to support the GPIO UARTs (ttyAMA devices).

You can verify by checking /etc/udev/rules.d/10-ConsolePi.rules for KERNEL=="ttyAMA[1-4]*", GOTO="TTYAMA-DEVS" near the top of the file (line 2).

If /etc/udev/rules.d/10-ConsolePi.rules does not exist, nothing needs to be done. If it does and the verification above indicates it needs to be updated you can either delete it (which will remove any current aliases, you'll be starting over, but the file will take on the new format). -- or -- if you want to avoid the need to rename the serial devices you've already set aliases for, you can...

  sudo mv /etc/udev/rules.d/10-ConsolePi.rules ~/ # move the file out of the rules dir (to your home dir)
  sudo cp /etc/ConsolePi/src/10-ConsolePi.rules /etc/udev/rules.d  # copy the new file template to the rules dir
  cat ~/10-ConsolePi.rules  # cat the old file contents.
  # Select and copy only the lines with aliases that were previously configured
  sudo nano /etc/udev/rules.d/10-ConsolePi.rules  # open the new rules file for editing
  # paste in the lines with aliases from the previous rules file, in the same section they existed previously
  # Be sure not to remove any of the existing comments/lines
  # repeat cat, copy, edit, paste as needed if content existed in multiple sections.
  consolepi-resetudev  # trigger/reload udev to update devs based on new file content.
  consolepi-showaliases  # This utility verifies All aliases set in the rules file also exist in ser2net and vice versa.

Configure /boot/config.txt

  # related snippet from /boot/config.txt

  #Enable Default UART (used to access this ConsolePi not externally)
  enable_uart=1

  # Enable Additional UARTs
  # dtoverlay=uart0,<param>=<val>
  # Params: txd0_pin                GPIO pin for TXD0 (14, 32 or 36 - default 14)
  #         rxd0_pin                GPIO pin for RXD0 (15, 33 or 37 - default 15)
  #         pin_func                Alternative pin function - 4(Alt0) for 14&15,
  #                                 7(Alt3) for 32&33, 6(Alt2) for 36&37

  # Enable uart 2 on GPIOs 0,1
  dtoverlay=uart2

  # Enable uart 3 on GPIOs 4,5
  dtoverlay=uart3

  # Enable uart 4 on GPIOs 8,9
  dtoverlay=uart4

  # Enable uart 5 on GPIOs 12,13
  dtoverlay=uart5

Configure /boot/cmdline.txt

# /boot/cmdline.txt

# The default UART is enabled for "inbound" access to this Pi, the pins are actually not accessible in my setup so they are not used at all
console=serial0,115200 console=tty1 root=PARTUUID=73aabb97-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

# the remaining UARTS configured for external access, Note that uart3 is not actually being used due to pin access
console=ttyAMA1,115200
console=ttyAMA2,115200
console=ttyAMA3,115200
console=ttyAMA4,115200

Configure ConsolePi.yaml

ConsolePi.yaml needs to include a TTYAMA: key (where TTYAMA is not indented, it should be at the same level as CONFIG or the optional OVERRIDES, HOSTS, and POWER keys)

Given the example above with 3 uarts enabled (technically 4 but the default UART is used for "inbound" access)

TTYAMA: [ttyAMA1, ttyAMA2, ttyAMA3]

The onboard UARTs will then showup in the consolepi-menu as ttyAMA#, you can then use the rename option to assign a friendly name and configure custom serial settings (i.e. change the baud used by the menu, the rename option will also add the device to ser2net using the next available TELNET port in the 7xxx range)

Power Control Setup

Refer to Power Control Setup

ZTP Orchestration

Refer to ZTP Orchestration

OVERRIDES:

Optional Overrides to prevent consolepi-upgrade from updating ConsolePi related system files

To Upgrade ConsolePi it's recommended to use the consolepi-upgrade command. This runs the install/upgrade script which on upgrade will verify some of the system configuration related to ConsolePi functionality. If you've made customizations to any of the system files ConsolePi initially configures, the upgrade script will backup the file (to /etc/ConsolePi/bak) and replace it. This may be undesired if you've made customizations, to prevent this from occurring simply create an empty file (doesn't technically have to be empty) with the same name as the file you want to prevent being modified by ConsolePi in '/etc/ConsolePi/overrides' (i.e. touch /etc/ConsolePi/overrides/dhcpcd.conf)

consolepi-upgrade is the preferred method, but you can run consolepi-sync described in Convenience Commands or alternatively simply do a git pull from within /etc/ConsolePi to Upgrade. However there is some risk, as references to system paths/symlinks etc. may have changed, and consolepi-sync doesn't automate any of the system changes that may be part of the upgrade.

List of files ConsolePi will verify and potentially update

Service Files

Overriding Service files not only turns off validation of the contents of the systemd unit file, but also the state of the service (enabled/disabled)

Optional override Variables in ConsolePi.yaml

ConsolePi.yaml also supports a few customizations via the optional OVERRIDES section. Examples of most of them should be in your configuration after the install (with the default values, so they are not overriding anything just in place as a reference).

A summary of available overrides:

Console Server

TELNET

Don't overlook consolepi-menu which supports remote ConsolePi discovery and provides a single launch point into any local and remote connections discovered

!! It is recommended to use consolepi-addconsole or the rn option in the menu to provide a consistent alias for each adapter/device. If more than one adapter is plugged in, they are assigned the root device names (ttyUSB#/ttyACM#) in the order they are detected by the kernel. The can occasionally change after reboot / or if an adapter is disconnected. Setting the aliases is the only way to ensure predictability.

Note: Some cheap a@# serial console adapters don't define serial #s, which is one of the attributes used to uniquely identify the adapter. The rename function/consolepi-addconsole command now support these adapters, but with caveats explained by the script when such a case is detected.

Note: the 8000/9000 range is always valid even if you are using an adapter specifically mapped to a port in the 7000 range. So if you plug in an adapter (ttyUSB) pre-mapped to port 7005, and it's the only adapter plugged in, it would also be available on port 8001

SSH / BlueTooth

The consolepi-menu command can be used to display a menu providing options for any locally connected USB to Serial adapters. In addition to any remotely connected USB to serial adapters connected to other remote ConsolePis discovered (either via mdns or the Cloud Sync function). When connecting to ConsolePi via bluetooth this menu launches automatically.

Note that when using bluetooth the menu is limited to local adapters and remotes found in the local-cache file. Connect via SSH for full remote functionality in the menu.

Serial Adapter connection options (baud, flow-control, data-bits, parity) are extracted from ser2net.conf by consolepi-menu. If there is an issue getting the data it falls back to the default of 9600 8N1 which can be changed in the menu (option c)

Convenience Commands:

There are a few convenience commands created for ConsolePi during the automated install

Upgrading ConsolePi

Use consolepi-upgrade to upgrade ConsolePi. Simply doing a git pull may occasionally work, but there are a lot of system files, symlinks, etc. outside of the ConsolePi folder that are occasionally updated, those changes are made via the upgrade script.

ConsolePi ensures packages related to ConsolePi are configured per your configuration. If you've made customizations the existing config will be backed up to the bak directory, and the config will be updated. Use the OVERRIDES function to override this behavior for files you don't want updated.

Tested Hardware / Software

Raspberry Pi

ConsolePi requires Python version >= 3.6, which means it now requires Buster. If running an older version of ConsolePi the last supported version is taggeed stretch-final, but reccomend creating a Buster Image to get the latest features.

ConsolePi Should work on all variants of the RaspberryPi and will work on other Linux systems including wsl (Windows Subsystem for Linux).

​ If you find a variant of the Rpi that does not work, create an "issue" to let me know. If I have one I'll test when I have time to do so

ConsolePi will also work on other Linux systems as described in Alternative Hardware Installs.

ConsolePi @ Work!

Have some good pics of ConsolePi in action? Let me know.

ConsolePi in action ConsolePi in action ConsolePi in action ConsolePi in action

CREDITS

ConsolePi utilizes a couple of other projects so Some Credit

  1. ser2net (cminyard)

    This project provides a proxy that allows telnet/tcp connections to be made to serial ports on a machine.

    https://sourceforge.net/projects/ser2net/

    https://github.com/cminyard/ser2net

  2. Others Available via optional Utilities Installer consolepi-extras or during consolepi-upgrade