2.1 Data Format Convergence

[agrignard]

Trying ot converge to a 2.1 standard

For now here is some table that pushes to cityIO (this list is maint to evolved) https://cityscope.media.mit.edu/CS_CityIO_Frontend/

• https://cityio.media.mit.edu/api/table/citymatrix => using version 1.0 (@popabczhang it would be nice (but I guess would involved lots of change in the GH scanning) to see if there is any way to send a 2.1 version

• https://cityio.media.mit.edu/api/table/grasbrook => suppose to use 2.1 but a little variation make it not compatible see issue #63

• https://cityio.media.mit.edu/api/table/cityscopejs_walk => same as grassbrook see issue #63

• https://cityio.media.mit.edu/api/table/cityIO_Gama => 2.1 compatible read from virtual_table modified in GAMA (with random value now) and resend to cityIO as new table called cityIO_Gama

[RELNO]

v2.1 docs does not refer anywhere to array of arrays: https://github.com/CityScope/CS_CityIO_Backend/wiki/Data-Format#grid2d-list CityScopeJS, python and other scanners/senders/readers of at least past ~1 year all use a list object, not an array of arrays.

I'm totally open to a new format if needed (though it will require refactoring of several readers/scanners) but that we'd have to agree on one -- What does i[0,1,2...n] mean?

[agrignard]

Ok I was wrong i referred to block and I should have refered to grid that indeed doesn't specified anything. However there is unconstistancy between the different table and I understand that python is sending this format for a while (so a change can impact on other soft depending on the python scan)

The only problem now is that when there is more that one value we put an array of array

[[1,0],[2,0],...,[3,0]];

and when there is only one value we just use an array

[1,0,......,0];

It might sounds as a detail but the way it is parsed it is a bit different.

With the array of an array is that we just have to know the size of the block N(using the header block) and then just iterate on the array and get the N elements

[RELNO]

per @agrignard request, https://github.com/CityScope/CS_Scanner_Python/commit/69654441e9aa89ed1954107e3e6e108599daf598 embeds the new/old API v2.1 for the CityScoPy scanner.

See working example here: https://cityio.media.mit.edu/api/table/grasbrook

Lingo in wiki is reflecting the current state of the standard: https://github.com/CityScope/CS_CityIO_Backend/wiki/cityIO-Data-Structure

Older tables will update over time.

[agrignard]

Awesome!

Even if there is maybe still some "convergence" issue (like the one we mention this morning on the mapping) we can close this issue for now!

Thanks for the quick fix and the wiki update!