search

tomnomnom / gron

Make JSON greppable!
MIT License
13.87k stars 328 forks source link

failed to form statements: invalid character 'ï' looking for beginning of value #121

Open blabut opened 7 months ago

blabut commented 7 months ago

With a freshly installed version of gron from Howebrew, I am getting the following error when trying to gron a json from a BloodHound output:

$ cat 20240325120043_users.json | gron
failed to form statements: invalid character 'ï' looking for beginning of value

I checked manually and the input json file does not contain the incriminated ï character. Is there any known cause / workaround to this issue ?

I am under MacOS Ventura 13.6.

Thanks a lot!

RossPatterson commented 7 months ago

You're almost certainly processing a UTF-8 document, but not processing it as UTF-8. See https://en.wikipedia.org/wiki/%C3%8F#Computing for an explanation of the mistake.

blabut commented 7 months ago

I understand it must indeed be related to encoding, but it does not make sense to me that I get this error while processing json output from a standard tool.

kbilleter commented 7 months ago

Stupid BoMs. Try something like

sed $'1s/^\uFEFF//' 20240325120043_users.json | gron