Wish List - 2019/2020/2021/2022/2023

tavinus commented 6 years ago

Already Implemented

v0.3.2 :

[x] Sort package list by package names
[x] Ignore opkg warnings
- opkg list-upgradable may print something like "Multiple packages (libpthread and libpthread) providing same name marked HOLD or PREFER. Using latest." within the list. This used to cause an error because it would be parsed as a package name.
- It was kind of rare though. I THINK it only happens on TRUNK/BETA versions that are very old.
[x] Use awk + printf to print an ASCII table of the updates (prettier)
v0.3.4 :
[x] Load info from /etc/openwrt_release when available
[x] Add more router info on email report (from openwrt_release)
[x] Detect if the system is a DISTRIB_RELEASE='SNAPSHOT' and show warning about upgrading those
[x] New/prettier HTML email (new font type, sizes and info table)

Pending

[ ] Colorized output (must be optional)
[ ] Implement Self-Update parameter/functionality (to upgrade from github master)
[ ] Implement tests (travis-ci maybe?)
[ ] Option to select / choose which updates to apply
- Very cool idea but adds a lot on complexity / parsing, may be postponed
- Already have a numbered list that can be used
[ ] Wiki Content
- [ ] Troubleshooting
  - [ ] The squashfs "problem" (read only, uses new space)
  - [ ] The SNAPSHOT "problem" (why those should not be upgraded)
- [ ] Install / Download
- [ ] Self-Update
- [ ] Basic Usage / Examples
- [ ] Emailing reports (ssmtp and others)
- [ ] Dealing with config file conflicts
[ ] Tools to compare / diff config files conflicts
- Should try to detect /etc/config/*-opkg files (or get from installer message)
- Use diff or git diff to offer comparison (if available)
- File Operations for conf files
  - Remove
  - Move
  - Edit / Merge

tav

sgeto commented 5 years ago

Colorized output would be great

Goro2030 commented 5 years ago

Can you sort the HTML output that is sent in email by package name?

tavinus commented 5 years ago

I have started to code some of the enhancements.

Current changes (v0.3.0) :

[x] Sort package list by package names
[x] Ignore opkg warnings
- opkg list-upgradable may print something like "Multiple packages (libpthread and libpthread) providing same name marked HOLD or PREFER. Using latest." within the list. This used to cause an error because it would be parsed as a package name.
- It was kind of rare though. I THINK it only happens on TRUNK/BETA versions that are very old.
[x] Use awk + printf to print an ASCII table of the updates (prettier)

@sgeto When you say colorized output, you mean on CLI right?
I don't usually care too much about colors on terminal (I don't even use zsh for example).
Do you have any examples to show me on such colorized output? I do know how to use color on terminal, even though I never did it on ash/BuysBox.

For now, the output is prettier just by formatting a table, eg:

# ./opkg-upgrade.sh -n

Simple OPKG Updater v0.3.0

Done | Ignoring package lists update
Done | Getting upgradable packages list

Packages available for upgrade: 73

+-----+--------------------------+----------------------------+----------------------------+
|   # | Package                  | Current                    | Update                     |
+-----+--------------------------+----------------------------+----------------------------+
|   1 | busybox                  | 1.28.3-3                   | 1.29.3-3                   |
|   2 | dropbear                 | 2017.75-5                  | 2017.75-8                  |
|   3 | firewall                 | 2018-05-19-30463d08-1      | 2018-12-06-14589c80-2      |
|   4 | fstools                  | 2018-05-23-dd02dad3-1      | 2018-09-29-29e53af0-3      |
|   5 | hostapd-common           | 2018-05-21-62566bc2-4      | 2018-05-21-62566bc2-6      |
|   6 | ip6tables                | 1.6.2-1                    | 1.8.2-2                    |
|   7 | iptables                 | 1.6.2-1                    | 1.8.2-2                    |
|   8 | iwinfo                   | 2018-05-18-e59f9253-1      | 2018-07-31-65b8333f-1      |
|   9 | jshn                     | 2018-04-12-6eff829d-1      | 2018-07-25-c83a84af-2      |
|  10 | kmod-ath                 | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  11 | kmod-ath10k              | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  12 | kmod-ath9k               | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  13 | kmod-ath9k-common        | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  14 | kmod-cfg80211            | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  15 | kmod-gpio-button-hotplug | 4.9.107-2                  | 4.14.82-2                  |
|  16 | kmod-ip6tables           | 4.9.107-1                  | 4.14.82-1                  |
|  17 | kmod-ipt-conntrack       | 4.9.107-1                  | 4.14.82-1                  |
|  18 | kmod-ipt-core            | 4.9.107-1                  | 4.14.82-1                  |
|  19 | kmod-ipt-nat             | 4.9.107-1                  | 4.14.82-1                  |
|  20 | kmod-lib-crc-ccitt       | 4.9.107-1                  | 4.14.82-1                  |
|  21 | kmod-mac80211            | 4.9.107+2017-11-01-9       | 4.14.82+4.19-rc5-1-1       |
|  22 | kmod-nf-conntrack        | 4.9.107-1                  | 4.14.82-1                  |
|  23 | kmod-nf-conntrack6       | 4.9.107-1                  | 4.14.82-1                  |
|  24 | kmod-nf-ipt              | 4.9.107-1                  | 4.14.82-1                  |
|  25 | kmod-nf-ipt6             | 4.9.107-1                  | 4.14.82-1                  |
|  26 | kmod-nf-nat              | 4.9.107-1                  | 4.14.82-1                  |
|  27 | kmod-nf-reject           | 4.9.107-1                  | 4.14.82-1                  |
|  28 | kmod-nf-reject6          | 4.9.107-1                  | 4.14.82-1                  |
|  29 | kmod-ppp                 | 4.9.107-1                  | 4.14.82-1                  |
|  30 | kmod-pppoe               | 4.9.107-1                  | 4.14.82-1                  |
|  31 | kmod-pppox               | 4.9.107-1                  | 4.14.82-1                  |
|  32 | kmod-slhc                | 4.9.107-1                  | 4.14.82-1                  |
|  33 | libblobmsg-json          | 2018-04-12-6eff829d-1      | 2018-07-25-c83a84af-2      |
|  34 | libip4tc                 | 1.6.2-1                    | 1.8.2-2                    |
|  35 | libip6tc                 | 1.6.2-1                    | 1.8.2-2                    |
|  36 | libiwinfo                | 2018-05-18-e59f9253-1      | 2018-07-31-65b8333f-1      |
|  37 | libjson-c                | 0.12.1-1                   | 0.12.1-2                   |
|  38 | libjson-script           | 2018-04-12-6eff829d-1      | 2018-07-25-c83a84af-2      |
|  39 | libpthread               | 1.1.19-1                   | 1.1.20-1                   |
|  40 | libubox                  | 2018-04-12-6eff829d-1      | 2018-07-25-c83a84af-2      |
|  41 | libubus                  | 2018-01-16-5bae22eb-1      | 2018-10-06-221ce7e7-1      |
|  42 | libuci                   | 2018-03-24-5d2bf09e-1      | 2018-08-11-4c8b4d6e-1      |
|  43 | libuclient               | 2017-11-02-4b87d831-1      | 2018-11-24-3ba74ebc-1      |
|  44 | libxtables               | 1.6.2-1                    | 1.8.2-2                    |
|  45 | luci                     | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  46 | luci-app-firewall        | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  47 | luci-base                | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  48 | luci-lib-ip              | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  49 | luci-lib-jsonc           | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  50 | luci-lib-nixio           | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  51 | luci-mod-admin-full      | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  52 | luci-mod-network         | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  53 | luci-mod-status          | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  54 | luci-mod-system          | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  55 | luci-proto-ipv6          | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  56 | luci-proto-ppp           | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  57 | luci-ssl                 | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  58 | luci-theme-bootstrap     | git-18.295.70765-c74d9c0-1 | git-18.340.83538-1b26c91-1 |
|  59 | mtd                      | 21                         | 23                         |
|  60 | netifd                   | 2018-05-30-a580028d-1      | 2018-11-26-dfa4ede4-1      |
|  61 | odhcp6c                  | 2018-04-29-5316d7f8-12     | 2018-07-14-67ae6a71-16     |
|  62 | odhcpd-ipv6only          | 1.7-1                      | 1.13-3                     |
|  63 | ppp                      | 2.4.7-12                   | 2.4.7-14                   |
|  64 | ppp-mod-pppoe            | 2.4.7-12                   | 2.4.7-14                   |
|  65 | procd                    | 2018-03-28-dfb68f85-1      | 2018-11-23-d6673547-1      |
|  66 | rpcd                     | 2018-08-16-41333abe-1      | 2018-11-22-c79ef222-1      |
|  67 | uboot-envtools           | 2018.03-1                  | 2018.03-2                  |
|  68 | ubus                     | 2018-01-16-5bae22eb-1      | 2018-10-06-221ce7e7-1      |
|  69 | ubusd                    | 2018-01-16-5bae22eb-1      | 2018-10-06-221ce7e7-1      |
|  70 | uci                      | 2018-03-24-5d2bf09e-1      | 2018-08-11-4c8b4d6e-1      |
|  71 | uclient-fetch            | 2017-11-02-4b87d831-1      | 2018-11-24-3ba74ebc-1      |
|  72 | uhttpd                   | 2018-08-23-b741dec3-1      | 2018-11-28-cdfc902a-1      |
|  73 | wpad-mini                | 2018-05-21-62566bc2-4      | 2018-05-21-62566bc2-6      |
+-----+--------------------------+----------------------------+----------------------------+

Proceed with upgrade? (Y/y to proceed) n

Cancelled by user!

~I will push it in a few minutes...~ v0.3.0 is available Cheers!

Goro2030 commented 5 years ago

I have started to code some of the enhancements.

I will push it in a few minutes... v0.3.0 is available Cheers!

WOW, that was QUICK! Thanks

Goro2030 commented 5 years ago

It works like a charm ( v 0.3.0 ) . The only weird thing is that I get this during the updates:

Configuring luci-app-upnp.
uci: Entry not found
uci: Entry not found
Configuring luci-app-dnscrypt-proxy.
uci: Entry not found
uci: Entry not found
Configuring luci-app-bcp38.
uci: Entry not found
uci: Entry not found
Configuring luci-app-transmission.
uci: Entry not found
uci: Entry not found
Configuring luci-theme-rosy.
uci: Entry not found
uci: Entry not found

tavinus commented 5 years ago

Found a minor problem with the new listing, not printing the first item to screen (email is fine). Fixing... Will push in a few minutes

tavinus commented 5 years ago

@Goro2030 That is weird... It seems to be a opkg post-install error (when applying the new configs).
Can you try to run those manually to see if the same problem happens?

Example:

opkg install luci-app-upnp

It may be that it only happens when upgrading though.

Goro2030 commented 5 years ago

@Goro2030 That is weird... It seems to be a opkg post-install error (when applying the new configs). Can you try to run those manually to see if the same problem happens?

Example:
opkg install luci-app-upnp
It may be that it only happens when upgrading though.

Yes, it happens anyway (when manually updating) . Offtopic, but, what can be wrong with my installation?

tavinus commented 5 years ago

@Goro2030 Found this: https://forum.openwrt.org/t/uci-entry-not-found-after-opkg-upgrade/7631

Quote:

found/fixed the issue I had some backup config files in /etc/config, after I removed this backup config files all uci: Entry not found errors are gone

tavinus commented 5 years ago

Version 0.3.4 was pushed.
Any feedback would be appreciated.

v0.3.4 :

[x] Load info from /etc/openwrt_release when available
[x] Add more router info on email report (from openwrt_release)
[x] Detect if the system is a DISTRIB_RELEASE='SNAPSHOT' and show warning about upgrading those
[x] New/prettier HTML email (new font type, sizes and info table)
[x] Code cleanup from v0.3.0 implementations

Goro2030 commented 5 years ago

v0.3.5 works perfectly. New looks of the html email sent is great!

sgeto commented 5 years ago

Sorry for the delay. The old OpenWRT wiki points to a wrapper/script here that can give colorized output. Unfortunately, the link became unavailable before I could have a look at it. But it should generally be possible to use ascii colors in shell scripts. If busybox's ls applet supports the --color option, then so can other others.

Something like this maybe (sorry for the terrible script):

#!/bin/sh

# Colors config . . . [[ JZ . . . ]]
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"

# Router Details
MTHOST="192.168.1.11"

# Check if Router is accessible. If not, EXIT immediately.
while ! ping -c 1 $MTHOST &>/dev/null
do
    echo -e ""$COL_RED"ALERT ..... Router $MTHOST is DOWN$COL_RESET"
    exit 1
done
echo -e "Router is accessible... $COL_GREEN OK $COL_RESET"

tavinus commented 5 years ago

Thanks for the snippet.

Since this is busybox/ash there is no tput to use (by default).
There is also no simple way to test if colors are available and I know for a fact that older versions of busybox do NOT support colors.

I made a little research online and adapted some stuff I tested within this little test script.

#!/bin/sh

# Gustavo Arnosti Neves - Dec 2018

# Modified from many other snippets
# Adapted to work virtually anywhere
# Works on busybox/ash

ESeq="\x1b["
RCol="$ESeq"'0m'    # Text Reset

# Regular               Bold                    Underline               High Intensity          BoldHigh Intens         Background              High Intensity Backgrounds
Bla="$ESeq"'0;30m';     BBla="$ESeq"'1;30m';    UBla="$ESeq"'4;30m';    IBla="$ESeq"'0;90m';    BIBla="$ESeq"'1;90m';   On_Bla="$ESeq"'40m';    On_IBla="$ESeq"'0;100m';
Red="$ESeq"'0;31m';     BRed="$ESeq"'1;31m';    URed="$ESeq"'4;31m';    IRed="$ESeq"'0;91m';    BIRed="$ESeq"'1;91m';   On_Red="$ESeq"'41m';    On_IRed="$ESeq"'0;101m';
Gre="$ESeq"'0;32m';     BGre="$ESeq"'1;32m';    UGre="$ESeq"'4;32m';    IGre="$ESeq"'0;92m';    BIGre="$ESeq"'1;92m';   On_Gre="$ESeq"'42m';    On_IGre="$ESeq"'0;102m';
Yel="$ESeq"'0;33m';     BYel="$ESeq"'1;33m';    UYel="$ESeq"'4;33m';    IYel="$ESeq"'0;93m';    BIYel="$ESeq"'1;93m';   On_Yel="$ESeq"'43m';    On_IYel="$ESeq"'0;103m';
Blu="$ESeq"'0;34m';     BBlu="$ESeq"'1;34m';    UBlu="$ESeq"'4;34m';    IBlu="$ESeq"'0;94m';    BIBlu="$ESeq"'1;94m';   On_Blu="$ESeq"'44m';    On_IBlu="$ESeq"'0;104m';
Pur="$ESeq"'0;35m';     BPur="$ESeq"'1;35m';    UPur="$ESeq"'4;35m';    IPur="$ESeq"'0;95m';    BIPur="$ESeq"'1;95m';   On_Pur="$ESeq"'45m';    On_IPur="$ESeq"'0;105m';
Cya="$ESeq"'0;36m';     BCya="$ESeq"'1;36m';    UCya="$ESeq"'4;36m';    ICya="$ESeq"'0;96m';    BICya="$ESeq"'1;96m';   On_Cya="$ESeq"'46m';    On_ICya="$ESeq"'0;106m';
Whi="$ESeq"'0;37m';     BWhi="$ESeq"'1;37m';    UWhi="$ESeq"'4;37m';    IWhi="$ESeq"'0;97m';    BIWhi="$ESeq"'1;97m';   On_Whi="$ESeq"'47m';    On_IWhi="$ESeq"'0;107m';

printInfo() {
        echo -e "${BICya}>>>> ${BIYel}${1}${RCol}"
}

printInfo "REGULAR"
echo -e "${Bla}BLACK ${Blu}BLUE ${Red}RED ${Gre}GREEN ${Yel}YELLOW ${Pur}PURPLE ${Cya}CYAN ${Whi}WHITE ${RCol}\n"

# Using printf
#printf "${Bla}BLACK ${Blu}BLUE ${Red}RED ${Gre}GREEN ${Yel}YELLOW ${Pur}PURPLE ${Cya}CYAN ${Whi}WHITE ${RCol}\n\n"

printInfo "BOLD"
echo -e "${BBla}BLACK ${BBlu}BLUE ${BRed}RED ${BGre}GREEN ${BYel}YELLOW ${BPur}PURPLE ${BCya}CYAN ${BWhi}WHITE ${RCol}\n"

printInfo "UNDERLINE"
echo -e "${UBla}BLACK ${UBlu}BLUE ${URed}RED ${UGre}GREEN ${UYel}YELLOW ${UPur}PURPLE ${UCya}CYAN ${UWhi}WHITE ${RCol}\n"

printInfo "HIGH INTENSITY"
echo -e "${IBla}BLACK ${IBlu}BLUE ${IRed}RED ${IGre}GREEN ${IYel}YELLOW ${IPur}PURPLE ${ICya}CYAN ${IWhi}WHITE ${RCol}\n"

printInfo "BOLD HIGH INTENSITY"
echo -e "${BIBla}BLACK ${BIBlu}BLUE ${BIRed}RED ${BIGre}GREEN ${BIYel}YELLOW ${BIPur}PURPLE ${BICya}CYAN ${BIWhi}WHITE ${RCol}\n"

printInfo "BACKGROUND"
echo -e "${On_Bla}BLACK ${On_Blu}BLUE ${On_Red}RED ${On_Gre}GREEN ${On_Yel}YELLOW ${On_Pur}PURPLE ${On_Cya}CYAN ${On_Whi}WHITE ${RCol}\n"

printInfo "HIGH INTENSITY BACKGROUND"
echo -e "${On_IBla}BLACK ${On_IBlu}BLUE ${On_IRed}RED ${On_IGre}GREEN ${On_IYel}YELLOW ${On_IPur}PURPLE ${On_ICya}CYAN ${On_IWhi}WHITE ${RCol}\n"

Example run:
(results may vary depending on term emulator)

Now I need to make hundreds of tests on how to make it pretty with colors.
Suggestions are welcome!

I will probably make a flag to disable colorized output as well, since some people dislike/disable colors.
Should probably just redefine all color vars to an empty string to disable colors.

I may also do it in a way that it can be themed, which should be almost the same effort:

Use custom variable names when printing
Redefine the variables used on prints to other colors
No colors = "empty" print variables

Cheers! Gus

tavinus / opkg-upgrade