No spaces allowed in any part of JSON read through tickParse!

[XLTechie]

I was doing some testing related to the problems discovered in issue #35 and found this.

If you try to tickParse some JSON, any JSON, that contains spaces anywhere in it (keys, values, or interstitially), you will get one of these errors:

Most serious: In a value

ticktick.sh: eval: line 424: unexpected EOF while looking for matching `"'
ticktick.sh: eval: line 425: syntax error: unexpected end of file

Interstitially or in a key, like in #35

ticktick.sh: line 424: TICKTICK: command not found

@kristopolous I'm pretty sure that this issue and #35 are related to some eval and quote expansion interaction somewhere.

• It is required that spaces be allowed in string values to JSON elements.
• We can work around the interstitial case by removing spaces outside quotes while tokenizing.
• I'm not sure if the JSON spec allows spaces in keys, but reading in JSON without tickParse (I.E. through the normal double backtick method) replaces the space with a number 32, which is fine.

[kristopolous]

hah ok it should definitely allow it

[kristopolous]

see 1d257919f9944ed2882d89f610c0b9b0d6e21fe7 for the tests

[kristopolous]

ah ok it breaks everything right ... isn't this lovely ... can we do unicode?

[kristopolous]

heh nope ... I bet kaos's bash array thing would work here. essentially I'll need to create a notion of "escaping" that avoids collision ... always fun.

[kristopolous]

I spend one evening creating a transpiler in bash 9 years ago ...

[XLTechie]

Hahhahhah!

I may take a run at fixing it later this weekend.

[kristopolous]

I actually already had put in code to handle it apparently ... it should be working ... so this is a bug, not an unimplemented feature.

[kristopolous]

ok yes, this makes things hard with the given approach, what a pain:

m  = ["a b"]

This space should be preserved.

m = {"a b": "c"}

This one is not.  

What about

m["a b"] ?

We need to now have a white-space ignoring look-behind of 2

I suppose I can support this basic syntax ...

[kristopolous]

alright, super close, just doing some testing

[XLTechie]

My question is this:

importing with:

something = { "the space": "should go here" }

Already works fine.

Can tickParse be refactored to use whatever mechanism the code scraping method is using?

I know how tickParse and the chain it calls work, but haven't read enough to figure out how the ... part does its thing.

[kristopolous]

nope. it's more complicated. I'm almost done

[kristopolous]

there, it was simpler than I first thought

[XLTechie]

Sorry, no.

Try this JSON:

{"string":"strin gy!","int":7,"array":["first","seco nd"]}

ticktick.sh: eval: line 424: unexpected EOF while looking for matching `"' ticktick.sh: eval: line 425: syntax error: unexpected end of file

And if you try this JSON:

{"string":"stringy!", "int":7, "array":["first","second"]}

ticktick.sh: line 424: TICKTICK: command not found string='stringy!'

That's actually issue #35 again, but fixing this should fix #35.

[XLTechie]

I think you might be able to replace that whole function body with something like this:

__tick_json_sanitize_value() { sed -r 's/[-\";,=()[]{}.'\'': ]/'\'\'/g }

Doing that, with this JSON:

{"string": "string'y!", "int":7, "array":["fir-st","sec;ond"]}

No longer gives the first set of errors above. It still fails because of #35 though. So I can't actually confirm that this is a fix for this issue, only that it still appears to work in cases where it worked before.

Remove the spaces from that, and the change gives:

array_000000000000=fir-st array_000000000001='sec;ond' int=7 string='string'\''y!'