stan-dev / stan

Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
2.57k stars 368 forks source link

json writer for structured data #3176

Closed mitzimorris closed 1 year ago

mitzimorris commented 1 year ago

Summary:

Add a JSON writer callback, allowing for output of structured information.

Description:

The current writer callback methods are only defined for atomic elements and vectors. The Stan services layer uses these callbacks to assemble inference outputs into CSV files. Any structured information must either flattened or output as comments which require custom, ad-hoc post-procesing.

The immediate use case for JSON output is to output diagnostics from the single-path Pathfinder algorithm - see https://github.com/stan-dev/stan/pull/3123 - https://github.com/stan-dev/stan/pull/3123#issuecomment-1490936361

More use cases are outlined in the https://github.com/stan-dev/design-docs/blob/master/designs/0032-stan-output-formats.md

To implement this we need to introduce a new writer class: stan::io::structured_writer which adds:

everywhere we need to keep track of how many of each entry because JSON is picky about trailing commas figure out where the best place to put this logic is.

so that we can create nested records.

Current Version:

v2.31.0