ABridoux / scout

Reading and writing in JSON, Plist, YAML and XML data made simple when the data format is not known at build time. Swift library and command-line tool.
https://www.woodys-findings.com/scout
MIT License
129 stars 6 forks source link

Conversion from one format to another #133

Closed ABridoux closed 3 years ago

ABridoux commented 4 years ago

Offer the possibility to export a data format into another one like Plist -> JSON, XML -> Yaml rather than only perform a CSV export.

haakonstorm commented 3 years ago

Wow, Plist2JSON support!

But, when plutil or plistbuddy reports invalid object in plist for destination format, will you be able to handle it somehow?

E.g: my test case with the output of the airport cli tool, buried in PrivateFrameworks:

# symlink the airport tool to preserve user sanity:

λ ln -s \
      /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport \
      /usr/local/bin/airport
# relevant usage for us for the airport tool

λ airport --help \
      | grep -E 'Supp|\-\-scan|\-\-xml'

Supported arguments:
 -s[<arg>] --scan=[<arg>]       Perform a wireless broadcast scan.
 -x        --xml                Print info as XML
# enumerate visible 80211 ssids as seen from this workstation:

λ airport --scan \
      | head -n5

                            SSID BSSID             RSSI CHANNEL HT CC SECURITY (auth/unicast/group)
                  Telenor7623mat 74:3a:ef:40:ef:47 -94  11      Y  NO WPA2(PSK/AES/AES)
                            web5 88:3d:24:70:ce:5a -93  11      Y  NL WPA2(PSK/AES/AES)
                  Telenor7623mat 74:3a:ef:40:ef:43 -90  100     Y  NO WPA2(PSK/AES/AES)
         TP-LINK_Extender_2.4GHz ec:08:6b:b3:c2:54 -89  2,+1    Y  -- NONE
# outputting as xml plist works, and now provides ssid details

λ (head -n5 && printf '\n ---- 8< ----\n\n' && tail -n5) \
      <<< $(airport --scan --xml)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
    <dict>

 ---- 8< ----

        <key>SSID_STR</key>
        <string>Caesar 5GHz</string>
    </dict>
</array>
</plist>
# but converting to JSON not so easy!

λ airport --scan --xml | plutil -convert json -- -

<stdin>: invalid object in plist for destination format

I'd love to be able to parse output like this, and other xml plists output from apple cli utilities, as JSON! Hope you manage to get this working somehow!

ABridoux commented 3 years ago

From what I understand, and after having run airport --scan --xml, I don't think there should be a problem to convert the output to a JSON format since the Plist seems valid. In fact, you can already use a tool like JSON Editor to convert a Plist to JSON, although this cannot be used in the terminal. To let you know, I will maybe work on those new features next week since I'll be on vacation.

haakonstorm commented 3 years ago

Thanks to you, - i had given up searching -, I found a JS tool that
I had given up searching, but thanks to you I found a JS tool that actually did the job correctly. yq / xq didn't.

npm install -g plist-to-json

This actually works, and is scriptable :) thnx!! *(EDIT: What I wrote could easily come across as I had given up searching for solutions and that @ABridoux was to blame for that 🤣. Rearranged words to un-douchebag my post.)

haakonstorm commented 3 years ago

Out of curiosity: you validated the output of airport -s -x as valid plist xml, but why do plutil then complain about invalid objects when called to convert the input to JSON?

λ airport --scan --xml | plutil -convert json -o - -- -

<stdin>: invalid object in plist for destination format

but:

λ airport --scan --xml | plist-to-json /dev/stdin | jq . | head -5

[
  [
    {
      "80211D_IE": {
        "IE_KEY_80211D_CHAN_INFO_ARRAY": [

Also, I have another question but I'll pop it in a separate issue as a feature-request! :-)

ABridoux commented 3 years ago

I think plutil does not handle properly the data values in the Plist. Once removed, it is able to convert to JSON.

haakonstorm commented 3 years ago

I mean to have read somewhere (source: the internet) that the problem plutil encounters is when date or time values are encountered in a property list.

Perhaps a utility feature scout could provide: some sort of duct-tape-type-pipe filter for macOS and macOS Server administration work - this is a place Apple's plist-fetish still flower in all its glory, but also a place where separate dev teams don't talk to each other -- as evidenced by the above.

I'm thinking something along the lines of

airport --scan --xml | scout --totherescue | plutil --no-convert --just --dosomethingwiththeplistnatively

There are (still) a lot of macOS [Server] administration work done around the planet. This would be very useful for a lot of those engineers. After Apple became the new Nokia, they have lost most if not all interest in real computers. I was an "Apple Certified Systems Administrator" they actually managed to tout as job-security-relevant. Couple of years later the discontinued first Final Cut Server, then the Apple Xserve RAID, then the Apple Xserve, which was all fantastic pieces of engineering, with Xsan storage area network and all, very much ahead of its time. Apple dropped everything of such, following the success of the Jesus Phone, and casually reccommended customers who had invested heavily in enterprise Apple infrastructure, to replace their servers with Mac Minis. Did you get to find some time to work on these translating/(transpiling?) features during your holiday? Personally, I've alternated between stuffing myself and being stuffed with food for 14 days straight now. Holidays are exhausting I need a holiday

ABridoux commented 3 years ago

Thanks to you, - i had given up searching -, I found a JS tool that

I had given up searching, but thanks to you I found a JS tool that actually did the job correctly. yq / xq didn't.


npm install -g plist-to-json

This actually works, and is scriptable :) thnx!!

*(EDIT: What I wrote could easily come across as I had given up searching for solutions and that @ABridoux was to blame for that 🤣. Rearranged words to un-douchebag my post.)

Ha ha don't worry it was clear that you were not a douchebad.

haakonstorm commented 3 years ago

Actually, pronounced in norwegian-french, douchebad comes across surprisingly positive!