Fix curl etag handling

[neilmartin83]

curl will output a 0 byte file to the --etag-save path if --etag-compare passes. From the curl manpage:

--etag-compare <file>
              (HTTP) This option makes a conditional HTTP request for the
              specific ETag read from the given file by sending a custom
              If-None-Match header using the stored ETag.

              For correct results, make sure that the specified file contains
              only a single line with the desired ETag. An empty file is
              parsed as an empty ETag.

              Use the option --etag-save to first save the ETag from a
              response, and then use this option to compare against the saved
              ETag in a subsequent request.

This causes the scripts to re-download the SOFA json file on every other re-run because the cached etag file (which is also the file being compared) is empty, so curl's --etag-compare fails with it.

Every re-run echos out the following even if curl did not re-download the SOFA json file:

Cached ETag did not match online ETag, so downloaded new SOFA json file

Curl will re-download the SOFA json file every other run (removed --silent flag to test):

% '/Users/neil.martin/Documents/GitHub/sofa/tool-scripts/XProtectVersionCheck-swiftDialog.sh'
e-tag stored, will download only if e-tag doesn't match
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Cached ETag did not match online ETag, so downloaded new SOFA json file

% '/Users/neil.martin/Documents/GitHub/sofa/tool-scripts/XProtectVersionCheck-swiftDialog.sh'
e-tag stored, will download only if e-tag doesn't match
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11732  100 11732    0     0   234k      0 --:--:-- --:--:-- --:--:--  238k
Cached ETag did not match online ETag, so downloaded new SOFA json file

This PR will ensure the following behaviour:

1. If macos_data_feed_etag_old.txt OR SOFA json file are not present, download the SOFA json file and store etag in macos_data_feed_etag_old.txt
2. If macos_data_feed_etag_old.txt AND the SOFA json file are present, curl and compare etag then save new etag to macos_data_feed_etag_new.txt - it will be 0 bytes if the comparison passes.
3. If macos_data_feed_etag_old.txt exists AND is 0 bytes, OR macos_data_feed_etag_old.txt matches macos_data_feed_etag_new.txt, a new json file was not downloaded, so we echo the correct output.
4. Otherwise, we did download a new SOFA json file, so we overwrite macos_data_feed_etag_old.txt with the contents of macos_data_feed_etag_new.txt

[grahampugh]

Thanks, this snippet seems to work, can we get the variable names as below?:

# local store
json_cache_dir="/private/tmp/sofa"
json_cache="$json_cache_dir/macos_data_feed.json"
etag_cache="$json_cache_dir/macos_data_feed_etag.txt"
etag_cache_temp="$json_cache_dir/macos_data_feed_etag_temp.txt"

# ensure local cache folder exists
/bin/mkdir -p "$json_cache_dir"

# check local vs online using etag (only available on macOS 12+)
if [[ -f "$etag_cache" && -f "$json_cache" ]]; then
    etag_old=$(/bin/cat "$etag_cache")
    /usr/bin/curl --compressed --silent --etag-compare "$etag_cache" --etag-save "$etag_cache_temp" --header "User-Agent: $user_agent" "$online_json_url" --output "$json_cache"
    etag_new=$(/bin/cat "$etag_cache_temp")
    if [[ "$etag_old" == "$etag_new" || $etag_new == "" ]]; then
        echo "Cached ETag matched online ETag - cached json file is up to date"
        /bin/rm "$etag_cache_temp"
    else
        echo "Cached ETag did not match online ETag, so downloaded new SOFA json file"
        /bin/mv "$etag_cache_temp" "$etag_cache"
    fi

[neilmartin83]

Done! And tested. Thank you.

[grahampugh]

@neilmartin83 I was thinking, it would be great to bump the user-agent to 1.1 in each script so that we could theoretically see who's running the new version of the scripts vs the old version.

e.g.

user_agent="SOFA-Jamf-EA-macOSCompatibilityCheck/1.1"

[neilmartin83]

@grahampugh - done!

[poundbangbash]

Just as another data point... I ran into similar issues where I had two EAs in Jamf running in one run, so it was effectvitly running the same caching code back to back. The recon resulted in an empty, zero-byte etag cache file.

I grabbed the code from Neil's PR, added two EAs to Jamf, and ran recon a couple times. The etag file remains populated with the etag value now.