Include release date on OS tab of Software page

[pintomi1989]

As a Fleet user, I would like to be able to quickly determine the release date of an OS version when looking under the "OS" tab of the Software page in Fleet. This could be presented as a new column, next to "Version".

[JoStableford]

Related to a Slack conversation

[noahtalerman]

Hey @pintomi1989, heads up, we didn't have room to bring this into the current design sprint (4.48).

Please feel free to bring this back to feature fest.

[noahtalerman]

@nonpunctual when you get the chance, can you please share the script that scrapes Apple in the issue here?

[noahtalerman]

@pintomi1989 we discussed this during the last feature fest.

We decided not to work on drafts for this in the upcoming sprint (4.49)

Removing from feature fest.

[pintomi1989]

Note: This is currently a nice to have, not high priority for customer-stazzema

[nonpunctual]

@noahtalerman I am not sure if this is the one you mean. This script is really dumb because I wrote it but the idea & the URL sources are here.

#!/bin/bash
#shellcheck disable=SC2128,SC2154,SC2178,SC2207

# email function
emlscpt(){
/usr/bin/python << "EOF"
# -*- coding: utf-8 -*-
import subprocess
import os
import sys
import time
import smtplib
import getpass
from email.mime.text import MIMEText
current_user = getpass.getuser()
today = time.strftime("%b %d, %Y")
email_address = ["willynilly@blah.com"]
# email_address = ["brockwalters@blah.com"]
txt = open('/private/tmp/workdir/eml.txt', 'r')
apps_updated_today = txt.readlines()
email_contents = "/private/tmp/workdir/email_message".format(current_user)
with open(email_contents, 'w') as fp:
    fp.write("Hello,\n\n")
    fp.write("Updates released:\n\n")
    for updated_app in apps_updated_today:
        fp.write("\t{}\n".format(updated_app))
    fp.write("You're welcome!\n\n")
    fp.write("ʕ•ᴥ•ʔ William Nilly | Senior Notification Bear | willynilly@group.blah.com | ")
with open(email_contents, 'rb') as fp:
    msg = MIMEText(fp.read())
msg['Subject'] = "NOTIFICATION: Software Updates"
msg['From'] = "willynilly@blah.com"
msg['To'] = ", ".join(email_address)
s = smtplib.SMTP('relay.blah.com')
s.sendmail("willynilly@blah.com", email_address, msg.as_string())
s.quit()
EOF
}

#reset variables

initvar(){
unset counter datmtch httpurl i
}

# variables
# http://mywiki.wooledge.org/BashPitfalls#month.3D.24.28date_.2B-.25m.29.3B_day.3D.24.28date_.2B-.25d.29
IFS=$'\n\t'
eval "$(/bin/date +'century="%C" year_00_Mon="%g" year_00_Sun="%y" year_Mon="%G" year_Sun="%Y" month_01="%m" month_abrv="%b" month_full="%B" week_00_Mon="%W" week_00_Sun="%U" week_01_Mon="%V" weekday_0_Sun="%w" weekday_1_Mon="%u" day_001="%j" day_01="%d" day_1="%e" day_abrv="%a" day_full="%A" hour_00="%H" hour_01="%I" hour_0="%k" hour_1="%l" minute_00="%M" second_00="%S" second_epoch="%s" ampm="%p" timezone_alpha="%Z" timezone_utc="%z"')"
dateDmY="$day_01 $month_abrv $year_Sun"                                             # 01 Jun 2020
datedmY="$(echo "$day_1"| /usr/bin/sed 's/^[[:space:]]//') $month_abrv $year_Sun"   # 1 Jun 2020
datedMY="$(echo "$day_1"| /usr/bin/sed 's/^[[:space:]]//') $month_full $year_Sun"   # 1 June 2020
datemdY="$month_abrv $(echo "$day_1"| /usr/bin/sed 's/^[[:space:]]//'), $year_Sun"  # Jun 1, 2020
dateYMD="${year_Sun}-${month_01}-${day_01}"                                         # 2020-06-01
timstmp="T${hour_00}:${minute_00}:${second_00}"                                     # T00:00:00
export dateDmY datedmY datedMY datemdY dateYMD timstmp
logpath='/Library/Logs/update.notify'
logging="$logpath/update.notify.log"
errtext='/Library/Logs/update.notify/xmllint.log'
workdir='/private/tmp/workdir'
crljson="$workdir/dat.json"
emltext="$workdir/eml.txt"

# create working directories and files
if ! [ -d "$logpath" ]
then
    /bin/mkdir -p "$logpath"
fi
echo "[INFO] $dateYMD$timstmp" >> "$errtext"
/bin/mkdir -p "$workdir" 
/bin/cat << "EOF" > "$crljson"
{"data":[{"product":"iOS|iPadOS|macOS|Safari|tvOS|watchOS|Xcode","URL":"https://support.apple.com/en-us/HT201222"},{"product":"macOS","name":["Big Sur","Catalina","Mojave"],"URL":"https://support.apple.com/en-us/HT201260"},{"product":"Apple Configurator 2 (macOS)","URL":"https://apps.apple.com/us/app/apple-configurator-2/id1037126344"},{"product":"Clips (iOS)","URL":"https://apps.apple.com/us/app/clips/id1212699939"},{"product":"Final Cut Pro (macOS)","URL":"https://apps.apple.com/us/app/final-cut-pro-x/id424389933?mt=12"},{"product":"GarageBand (iOS)","URL":"https://apps.apple.com/us/app/garageband/id408709785?mt=8"},{"product":"GarageBand (macOS)","URL":"https://apps.apple.com/us/app/garageband/id682658836?mt=12"},{"product":"iMovie (iOS)","URL":"https://apps.apple.com/us/app/imovie/id377298193?mt=8"},{"product":"iMovie (macOS)","URL":"https://apps.apple.com/us/app/imovie/id408981434?mt=12"},{"product":"iTunes Movie Trailers (iOS)","URL":"https://apps.apple.com/us/app/itunes-movie-trailers/id471966214?mt=8"},{"product":"iTunes U (iOS)","URL":"https://apps.apple.com/us/app/itunes-u/id490217893"},{"product":"Keynote (iOS)","URL":"https://apps.apple.com/us/app/keynote/id361285480?mt=8"},{"product":"Keynote (macOS)","URL":"https://apps.apple.com/us/app/keynote/id409183694?mt=12"},{"product":"Logic Pro (macOS)","URL":"https://apps.apple.com/us/app/logic-pro-x/id634148309?mt=12"},{"product":"Microsoft Excel (macOS)","URL":"https://apps.apple.com/us/app/microsoft-excel/id462058435?mt=12"},{"product":"Microsoft PowerPoint (macOS)","URL":"https://apps.apple.com/us/app/microsoft-powerpoint/id462062816?mt=12"},{"product":"Microsoft Word (macOS)","URL":"https://apps.apple.com/us/app/microsoft-word/id462054704?mt=12"},{"product":"Music Memos (iOS)","URL":"https://apps.apple.com/us/app/music-memos/id1036437162"},{"product":"Numbers (iOS)","URL":"https://apps.apple.com/us/app/numbers/id361304891?mt=8"},{"product":"Numbers (macOS)","URL":"https://apps.apple.com/us/app/numbers/id409203825?mt=12"},{"product":"Pages (iOS)","URL":"https://apps.apple.com/us/app/pages/id361309726?mt=8"},{"product":"Pages (macOS)","URL":"https://apps.apple.com/us/app/pages/id409201541?mt=12"}]}
EOF

# parse "Apple security updates" https://support.apple.com/en-us/HT201222 (index 0 in json)
# capture each column of each row in the table on the page as an array member of "${datmtch[@]}"
# products should always be in the 1st column, dates should always be in the 3rd column
# if the column holds a date, check that the year matches the current year (no need to check previous years)
# if the column holds a date, check if the 1st column matches a product category
# if a product category matches, check to see if the date matches today's date
# if the date matches today's date, add the product info in column 1 to the "${updater[@]}" array
# unset the "row" array & repeat to process the next row in the table
httpurl=$(/usr/local/bin/jq -r ".data[0].URL" "$crljson")
datmtch=($(/usr/bin/curl -sS "$httpurl" | /usr/bin/sed -n '/\<tbody\>/,/\<\/tbody\>/p' | /usr/bin/sed 's/\<\/table\>//' | /usr/bin/xmllint -format -html - 2>> "$errtext" | /usr/bin/sed -n '/\<td\>/p;/\<p\>/p;' | /usr/bin/sed 's/<[^>]*>//g;s/&nbsp;/ /g;/^[[:space:]]*$/d;1,3d'))
counter="${#datmtch[@]}"
for ((i=0;i<"$counter";i+=1))
{
    tblrows+=("${datmtch[i]}")
    if [[ "${datmtch[i]}" =~ ^[0-9]{1,2}[[:space:]]{1}[A,D,F,J,M,N,O,S]{1}[a-z]*[[:space:]]{1}2[0-9]{3}$ ]]
    then
        yearchk=$(echo "${datmtch[i]}" | /usr/bin/awk '{print $3}')
        if [ "$yearchk" != "$year_Sun" ]
        then
            break
        fi
        if echo "${tblrows[0]}" | /usr/bin/grep -E -iq -e '^iOS' -e '^iPadOS' -e '^macOS' -e 'Big Sur' -e 'Catalina' -e 'Mojave' -e 'High Sierra' -e '^Safari' -e '^tvOS' -e '^watchOS' -e '^Xcode' && echo "${datmtch[i]}" | /usr/bin/grep -E -iq "$dateDmY|$datedmY|$datedMY"
        then
            updater+=("${tblrows[0]}")
        fi
        unset "tblrows"
    fi
}
initvar

# TEST DATE if [ "$datmtch" = "Jul 9, 2019" ]
# parse https://apps.apple.com/us/app/... (starting @ index 2 in json)
# scrape the product page for the release date
# if the date matches today's date add the product to the "${updater[@]}" array
counter=$(/usr/local/bin/jq '.data | length' "$crljson")
for ((i=2;i<"$counter";i+=1))
{
    datmtch=$(/usr/bin/curl -sS "$(/usr/local/bin/jq -r ".data[$i].URL" "$crljson")" | /usr/bin/awk '/time data-test-we-datetime/{print $3}' | /usr/bin/sed 's/datetime=//;s/"//g;s/.000Z//g' | { read -r var ; /bin/date -j -f "%Y-%m-%dT%H:%M:%S" "$var" "+%b %e, %Y"; } | /usr/bin/sed 's/  / /g')
    if [ "$datmtch" = "$datemdY" ]
    then
        updater+=("$(/usr/local/bin/jq -r ".data[$i].product" "$crljson")")
    fi
}
initvar

# prevent duplicate notifications
for j in "${!updater[@]}"
do
    arrmmbr="$(echo "${updater[j]}" | /usr/bin/sed 's/(/\\(/g;s/)/\\)/g')"
    if /usr/bin/awk "/$arrmmbr/&&/$dateYMD/{print}" "$logging" | read -r
    then
        echo "[INFO] preventing duplicate notification for ${updater[j]}" >> "$logging"
        unset "updater[j]"
    fi
done

# any products not eliminated by the duplicate notification check will be a member of the "${updater[@]}" array
# the emlscpt function prints the contents of the "${updater[@]}" array, sends an email listing the products & logs the result
if [ "${#updater[@]}" -gt 0 ]
then
    for k in "${updater[@]}"
    do
        echo "$k" >> "$emltext"
        echo "[ALERT] $dateYMD$timstmp  $k updated!" >> "$logging"
    done
    emlscpt
else
    echo "[INFO] $dateYMD$timstmp   no updates." >> "$logging"
fi
/bin/rm -rf "$workdir"

[nonpunctual]

@noahtalerman @marko-lisica https://macadmins.github.io/sofa/ - Simple Organized Feed for Apple Software Updates