tomnomnom / gron

Make JSON greppable!
MIT License
13.73k stars 325 forks source link

Feature request: rename array members by using member property as "ID" #80

Closed haakonstorm closed 2 years ago

haakonstorm commented 3 years ago

JSONdot notation associative array of sorts

I've wanted to request this feature for at least six months now. But, as it is so simple in nature, I primarily looked to solve it with jq directly but failed. I resorted to solving it with a two-pass gron job + sed, but I see now this feature can be of great use to many other use cases.

Given the following input JSON

(exerpt snapshot of my Home Assistant instance, via hass-cli):

[
  {
    "entity_id": "person.storm",
    "state": "home",
    "attributes": {
      "editable": true,
      "id": "0a71abd732aa4c37aaa4c77ec4a0e592",
      "latitude": 10.65601134689769,
      "longitude": 10.65601134689769,
      "gps_accuracy": 65,
      "source": "device_tracker.sfonx",
      "user_id": "6077e1d6074c4c8a9103d88b9885ea3a",
      "friendly_name": "STORM",
      "entity_picture": "/api/image/serve/4750d81a3c4b566c6587d6e751bc49bd/512x512"
    },
    "last_changed": "2020-11-15T00:28:41.896545+00:00",
    "last_updated": "2020-11-16T12:31:30.635576+00:00",
    "context": {
      "id": "362a2fd22e4e415054af37b908c75fc4",
      "parent_id": null,
      "user_id": null
    }
  },
  {
    "entity_id": "sun.sun",
    "state": "below_horizon",
    "attributes": {
      "next_dawn": "2020-11-17T06:28:15+00:00",
      "next_dusk": "2020-11-17T15:36:21+00:00",
      "next_midnight": "2020-11-16T23:02:25+00:00",
      "next_noon": "2020-11-17T11:02:18+00:00",
      "next_rising": "2020-11-17T07:16:58+00:00",
      "next_setting": "2020-11-17T14:47:38+00:00",
      "elevation": -20.86,
      "azimuth": 267.49,
      "rising": false,
      "friendly_name": "SUN"
    },
    "last_changed": "2020-11-16T14:49:40.025488+00:00",
    "last_updated": "2020-11-16T17:37:44.031768+00:00",
    "context": {
      "id": "6298c72b78183a753678e76825d6375e",
      "parent_id": null,
      "user_id": null
    }
  },
  {
    "entity_id": "entity_controller.motion_kitchen",
    "state": "blocked",
    "attributes": {
      "sensor_type": "duration",
      "mode": "night",
      "delay": "600s",
      "last_triggered_by": "binary_sensor.pr_kitchen",
      "blocked_at": "2020-11-16T18:50:21.038561",
      "blocked_by": "light.stove",
      "friendly_name": "motion_kitchen",
      "icon": "mdi:close-circle"
    }
  },
  {
    "entity_id": "light.bedroom_mini",
    "state": "off",
    "attributes": {
      "is_deconz_group": false,
      "friendly_name": "Mini sov",
      "supported_features": 41
    },
    "last_changed": "2020-11-16T12:00:46.878946+00:00",
    "last_updated": "2020-11-16T12:00:46.878946+00:00",
    "context": {
      "id": "a14e69bf8b96e4ae847d114c2c7c3384",
      "parent_id": null,
      "user_id": null
    }
  }
]

Stuffing it through a shiny gron pipe, since it is an array, gron yields:

json = [];
json[0] = {};
json[0].attributes = {};
json[0].attributes.editable = true;
json[0].attributes.entity_picture = "/api/image/serve/4750d81a3c4b566c6587d6e751bc49bd/512x512";
json[0].attributes.friendly_name = "STORM";
json[0].attributes.gps_accuracy = 65;
json[0].attributes.id = "0a71abd732aa4c37aaa4c77ec4a0e592";
json[0].attributes.latitude = 10.65601134689769;
json[0].attributes.longitude = 10.65601134689769;
json[0].attributes.source = "device_tracker.sfonx";
json[0].attributes.user_id = "6077e1d6074c4c8a9103d88b9885ea3a";
json[0].context = {};
json[0].context.id = "362a2fd22e4e415054af37b908c75fc4";
json[0].context.parent_id = null;
json[0].context.user_id = null;
json[0].entity_id = "person.storm";
json[0].last_changed = "2020-11-15T00:28:41.896545+00:00";
json[0].last_updated = "2020-11-16T12:31:30.635576+00:00";
json[0].state = "home";
json[1] = {};
json[1].attributes = {};
json[1].attributes.azimuth = 267.49;
json[1].attributes.elevation = -20.86;
json[1].attributes.friendly_name = "SUN";
json[1].attributes.next_dawn = "2020-11-17T06:28:15+00:00";
json[1].attributes.next_dusk = "2020-11-17T15:36:21+00:00";
json[1].attributes.next_midnight = "2020-11-16T23:02:25+00:00";
json[1].attributes.next_noon = "2020-11-17T11:02:18+00:00";
json[1].attributes.next_rising = "2020-11-17T07:16:58+00:00";
json[1].attributes.next_setting = "2020-11-17T14:47:38+00:00";
json[1].attributes.rising = false;
json[1].context = {};
json[1].context.id = "6298c72b78183a753678e76825d6375e";
json[1].context.parent_id = null;
json[1].context.user_id = null;
json[1].entity_id = "sun.sun";
json[1].last_changed = "2020-11-16T14:49:40.025488+00:00";
json[1].last_updated = "2020-11-16T17:37:44.031768+00:00";
json[1].state = "below_horizon";
json[2] = {};
json[2].attributes = {};
json[2].attributes.blocked_at = "2020-11-16T18:50:21.038561";
json[2].attributes.blocked_by = "light.stove";
json[2].attributes.delay = "600s";
json[2].attributes.friendly_name = "motion_kitchen";
json[2].attributes.icon = "mdi:close-circle";
json[2].attributes.last_triggered_by = "binary_sensor.pr_kitchen";
json[2].attributes.mode = "night";
json[2].attributes.sensor_type = "duration";
json[2].entity_id = "entity_controller.motion_kitchen";
json[2].state = "blocked";
json[3] = {};
json[3].attributes = {};
json[3].attributes.friendly_name = "Mini sov";
json[3].attributes.is_deconz_group = false;
json[3].attributes.supported_features = 41;
json[3].context = {};
json[3].context.id = "a14e69bf8b96e4ae847d114c2c7c3384";
json[3].context.parent_id = null;
json[3].context.user_id = null;
json[3].entity_id = "light.bedroom_mini";
json[3].last_changed = "2020-11-16T12:00:46.878946+00:00";
json[3].last_updated = "2020-11-16T12:00:46.878946+00:00";
json[3].state = "off";

Essentially, ultimately, what we`re looking for is output like this:

person.storm = "home";
person.storm.attributes.editable = true;
person.storm.attributes.entity_picture = "/api/image/serve/4750d81a3c4b566c6587d6e751bc49bd/512x512";
person.storm.attributes.friendly_name = "STORM";
person.storm.attributes.gps_accuracy = 65;
person.storm.attributes.id = "0a71abd732aa4c37aaa4c77ec4a0e592";
person.storm.attributes.latitude = 10.65601134689769;
person.storm.attributes.longitude = 10.65601134689769;
person.storm.attributes.source = "device_tracker.sfonx";
person.storm.attributes.user_id = "6077e1d6074c4c8a9103d88b9885ea3a";
person.storm.context.id = "362a2fd22e4e415054af37b908c75fc4";
person.storm.context.parent_id = null;
person.storm.context.user_id = null;
person.storm.last_changed = "2020-11-15T00:28:41.896545+00:00";
person.storm.last_updated = "2020-11-16T12:31:30.635576+00:00";
sun.sun = "below_horizon";
sun.sun.attributes.azimuth = 267.49;
sun.sun.attributes.elevation = -20.86;
sun.sun.attributes.friendly_name = "SUN";
sun.sun.attributes.next_dawn = "2020-11-17T06:28:15+00:00";
sun.sun.attributes.next_dusk = "2020-11-17T15:36:21+00:00";
sun.sun.attributes.next_midnight = "2020-11-16T23:02:25+00:00";
sun.sun.attributes.next_noon = "2020-11-17T11:02:18+00:00";
sun.sun.attributes.next_rising = "2020-11-17T07:16:58+00:00";
sun.sun.attributes.next_setting = "2020-11-17T14:47:38+00:00";
sun.sun.attributes.rising = false;
sun.sun.context.id = "6298c72b78183a753678e76825d6375e";
sun.sun.context.parent_id = null;
sun.sun.context.user_id = null;
sun.sun.last_changed = "2020-11-16T14:49:40.025488+00:00";
sun.sun.last_updated = "2020-11-16T17:37:44.031768+00:00";
sun.sun.state = "below_horizon";
entity_controller.motion_kitchen = "blocked";
entity_controller.motion_kitchen.attributes.blocked_at = "2020-11-16T18:50:21.038561";
entity_controller.motion_kitchen.attributes.blocked_by = "light.stove";
entity_controller.motion_kitchen.attributes.delay = "600s";
entity_controller.motion_kitchen.attributes.friendly_name = "motion_kitchen";
entity_controller.motion_kitchen.attributes.icon = "mdi:close-circle";
entity_controller.motion_kitchen.attributes.last_triggered_by = "binary_sensor.pr_kitchen";
entity_controller.motion_kitchen.attributes.mode = "night";
entity_controller.motion_kitchen.attributes.sensor_type = "duration";
entity_controller.motion_kitchen.state = "blocked";
light.bedroom_mini = "off";
light.bedroom_mini.attributes.friendly_name = "Mini sov";
light.bedroom_mini.attributes.is_deconz_group = false;
light.bedroom_mini.attributes.supported_features = 41;
light.bedroom_mini.context.id = "a14e69bf8b96e4ae847d114c2c7c3384";
light.bedroom_mini.context.parent_id = null;
light.bedroom_mini.context.user_id = null;
light.bedroom_mini.entity_id = "light.bedroom_mini";
light.bedroom_mini.last_changed = "2020-11-16T12:00:46.878946+00:00";
light.bedroom_mini.last_updated = "2020-11-16T12:00:46.878946+00:00";

The important change is renaming json[n] to the value of json[n].entity_id, which is guaranteed to be unique.

If the lone json[n] object itself can get the value of json[n].state, gron does 99% of the job.

If gron can attempt to flatten the the nested arrays, gron does 100% of the job:

json[0] = {};
(...)
json[0].state = "home";
json[0].entity_id = "person.storm";
json[0].last_changed = "2020-11-15T00:28:41.896545+00:00";

becoming:

person.storm = "home"
(...)
# perhaps keep this: person.storm.state = "home";
person.storm.last_changed = "2020-11-15T00:28:41.896545+00:00";

gron somehow needs to know it is getting a (nested) array, and where to get the name of the index.

E.g.:

gron --dotnotationableizewith="entity_id" --toplevelvaluesfrom="state" myjson.txt

Screenshot of iTerm2 (16-11-2020, 19-54-04)

This makes it easy to automate stuffs in your life by just writing shell scripts that you can execute arbitrarily. Like turning stuff on and off. But also by appending arbitrary commands to these scripts themselves. E.g. /light/bedroom_mini/turn_on.sh can have simple if-else: If my SO + myself aren't home, motion sensor tries to turn on lights, then it was the cat, and he prefer dark mode.

What do you think?

This project of mine has actually expanded to something much greater in scope, but this has been long enough already, I'll stop here. Thank you for your great work and success with gron!

hitzhangjie commented 3 years ago

I need this feature, too. Let's show a shorter and simpler example. Maybe it's easier to be understood.

My example is, I want to grep the object info whose field retParams contains a field cover, but theses objects are arranged in an array. So when I run gron $file | grep "cover", I got this.

json.nodes[6].retParams[25] = "cover";
json.nodes[21].retParams[10] = "cover";
json.nodes[22].retParams[1] = "cover";
json.nodes[40].retParams[7] = "cover";
json.nodes[43].retParams[2] = "cover";
json.nodes[94].retParams[26] = "cover";

This output is useless for me. You see index (json.nodes[94]) is large, more than 90. How could I locate the object in the $file.

I hope, we can display json.nodes[idx].$Attribute, like the attribute we used is Name, so the output will be:

Manner 1:

${nodeName2}.retParams[1]="cover"
${nodeName6}.retParams[25]="cover"
...

Manner 2:

Or, we can use other methods, like:

json.nodes[6].Name json.nodes[6].retParams[25] = "cover" ;
json.nodes[21].Name json.nodes[21].retParams[10] = "cover";
...

In this manner, we only prepend each line with a specific attibute value.