ZINC v3.0 support

[sjlongland]

New types:

XStr: Type("value")
List: [1, 2, 3]
Dict: {dis:"Building" site area:35000ft²}
Grid: <<ver:"3.0" ... >>

http://www.project-haystack.org/doc/Zinc

[sjlongland]

First cab off the rank will be lists… to be done by VRT as internal user story WC-1173.

[sjlongland]

Okay, so what's done:

• XStr data type
• Parsing of XStr, Dict, List and Grid in ZINC
• Parsing of Dict, List and Grid in JSON
• Dumping of Lists for JSON and ZINC (part of WC-1173)
• Unit tests relating to lists (WC-1173)

TO DO:

• Parsing of XStr in JSON
• Dumping of all of the above except lists
• Unit tests for dumping and parsing

[anyaelena]

Hello! I am working with the pyHaystack module and SkySpark for a load forecasting project. I need to parse a result grid containing Dicts and this is throwing parse errors. This is with hszinc 1.2.2 and pyHaystack 0.92.7. I can provide additional details but it looks like this may be as straightforward as accessing the changes referenced above. I'd appreciate advice on how to approach this.

Thank you!

[sjlongland]

Well, right now the code is far from complete, so there's no changes really to access. I think there have been some changes since this work started, so the code will need rebasing off current develop at best.

It's a TO-DO item, but basically it'll wait on the back-log until I get some spare time (a scarce resource lately -- practically working two jobs means I get almost no personal time to work on this stuff), or until my workplace has a need for it which would require WideSky to support these data types.

[anyaelena]

Thanks for the quick response @sjlongland. I'm going to hack something together on our end to get a MVP deployed w/out Dicts in the next week or so. (My colleague will be presenting this at Haystack Connect in May if you'll be there.)

Longer term, we do need Dicts. Are you interested in contributions? I can definitely justify spending a little time on this as the alternatives are to roll my own Python API wrapper or to jump out of Python for the API communication.

[sjlongland]

On 30/3/19 8:45 am, Anya Petersen wrote:

Thanks for the quick response @sjlongland https://github.com/sjlongland. I'm going to hack something together on our end to get a MVP deployed w/out Dicts in the next week or so. (My colleague will be presenting this at Haystack Connect in May if you'll be there.)

Yeah, no I don't do air travel (various reasons) so won't be at Haystack Connect.

Longer term, we do need Dicts. Are you interested in contributions? I can definitely justify spending a little time on this as the alternatives are to roll my own Python API wrapper or to jump out of Python for the API communication.

We certainly can take contributions. It's best if there's some test cases with them, but we'd be happy to review an implementation.

[anyaelena]

Sounds good. i'll reach back out assuming i can carve out the time to put together a respectable PR.

[dsimperiale]

Were you able to get this implemented? I stumbled up on this repository because I am also running into issues with dicts. When we installed/enabled the new dataquality pod from skyfoundry ( https://stackhub.org/package/sfxDataQualityExt )it creates a tag called hisDQMode which is a dictionary.

[anyaelena]

@dsimperiale Not me. We ran into another issue using pyHaystack (I think it was writing {forecast} data) and opted to go another route for integration. Might revisit in the future.

[sjlongland]

On 28/1/20 6:01 am, dsimperiale wrote:

Were you able to get this implemented? I stumbled up on this repository because I am also running into issues with dicts. When we installed/enabled the new dataquality pod from skyfoundry ( https://stackhub.org/package/sfxDataQualityExt )it creates a tag called hisDQMode which is a dictionary.

Not at this time, otherwise it'd be in master/develop branches, and this issue would be closed.

It's on the back-log, but I have no clue when I'll get around to doing it as I have no access to a server that implements dicts in order to test implementations.