tomnomnom / gron

Make JSON greppable!
MIT License
13.73k stars 325 forks source link

Feature request #86

Open Albonycal opened 3 years ago

Albonycal commented 3 years ago

Yesterday I was testing intel_gpu_top utility in linux which gives some information & has an option to output json with -J flag I wanted to write a script using this functionality using gron.. That utility constantly outputs in json which looks like this:

    "period": {
        "duration": 1000.323144,
        "unit": "ms"
    },
    "frequency": {
        "requested": 649.790024,
        "actual": 649.790024,
        "unit": "MHz"
    },
    "interrupts": {
        "count": 8.997093,
        "unit": "irq/s"
    },
    "rc6": {
        "value": 85.332937,
        "unit": "%"
    },
    "power": {
        "value": 0.192778,
        "unit": "W"
    },
    "engines": {
        "Render/3D/0": {
            "busy": 0.000000,
            "sema": 0.000000,
            "wait": 0.000000,
            "unit": "%"
        },
        "Blitter/0": {
            "busy": 0.000000,
            "sema": 0.000000,
            "wait": 0.000000,
            "unit": "%"
        },
        "Video/0": {
            "busy": 0.000000,
            "sema": 0.000000,
            "wait": 0.000000,
            "unit": "%"
        }
    }
}

My problem with is that it stops after reading the first value outputted by the tool. then the process stops... I want it to constantly show the output.. Example:

json = {};
json.engines = {};
json.engines["Blitter/0"] = {};
json.engines["Blitter/0"].busy = 0.000000;
json.engines["Blitter/0"].sema = 0.000000;
json.engines["Blitter/0"].unit = "%";
json.engines["Blitter/0"].wait = 0.000000;
json.engines["Render/3D/0"] = {};
json.engines["Render/3D/0"].busy = 0.000000;
json.engines["Render/3D/0"].sema = 0.000000;
json.engines["Render/3D/0"].unit = "%";
json.engines["Render/3D/0"].wait = 0.000000;
json.engines["Video/0"] = {};
json.engines["Video/0"].busy = 0.000000;
json.engines["Video/0"].sema = 0.000000;
json.engines["Video/0"].unit = "%";
json.engines["Video/0"].wait = 0.000000;
json.frequency = {};
json.frequency.actual = 0.000000;
json.frequency.requested = 0.000000;
json.frequency.unit = "MHz";
json.interrupts = {};
json.interrupts.count = 0.000000;
json.interrupts.unit = "irq/s";
json.period = {};
json.period.duration = 0.007298;
json.period.unit = "ms";
json.power = {};
json.power.unit = "W";
json.power.value = 0.000000;
json.rc6 = {};
json.rc6.unit = "%";
json.rc6.value = 0.000000;

It formats the output only one time and the stops but I want to do it constantly Is their some shell trick that can be used to do this (or a new feature) Thank you @albonycal

cho-m commented 3 years ago

Disclaimer: I'm not a shell expert, but I'll give some options I can think of.

Based on gron's current features, one possible option would be to process each separate JSON object into a single line and then use gron's streaming feature.

I can't comment on if this will work for intel_gpu_top, but the consolidation could be done with something like jq -c option for something like:

<process generating output> | jq --unbuffered -c '.' | gron -s

This will represent each JSON object as a value inside a JSON array, so a continuously running process can lead to very large number array index.

If you don't want the wrapping array, another possible option would be to send jq output to a loop that would repeatedly run gron. There may be some issues in my exact example, but hopefully the general gist comes across

<process generating output> | jq --unbuffered -c '.' | while read -r line; do echo "$line" | gron; done

Not too sure if there is a shell trick to restart gron after it exits. You will need to wait for someone else to chime in on that.


For gron feature, that doesn't rely on jq, it sounds like the option 2 that was not implemented from https://github.com/tomnomnom/gron/issues/23#issuecomment-259271490:

I've been giving some thought about the approach needed for this, and there's probably only two sane options:

  1. Require that the input be one JSON blob per line so it's easy to split on \n
  2. Do a pre-parse step to detect multiple JSON objects in the input

Option 1 is the implemented gron -s feature previously discussed.

Albonycal commented 3 years ago

nah it doesn't work.. anyway.. I stopped working on the script

dotcs commented 2 years ago

Not sure if I understood the question right, but maybe watch might help here?

E.g. place this script in /tmp/script.sh and give it executable rights. The script reads the uptime of the computer and creates a simple JSON document with the help of jq. Of course any additional steps, such as filtering the output and re-creating a JSON document with ungron could be done here as well.

#!/bin/bash
jq -c --null-input --arg uptime $(uptime | awk -F' ' '{ print $1 }') '{ uptime: $uptime }' | gron

Then run watch -n 1 /tmp/script.sh. The output is updated every 1 second.