Preserve headers order by default in CSV serialisation

bhftbootcamp / Serde.jl

Serde is a Julia library for (de)serializing data to/from various formats. The library offers a simple and concise API for defining custom (de)serialization behavior for user-defined types

Apache License 2.0

31 stars 7 forks source link

Preserve headers order by default in CSV serialisation #43

Closed dmitrii-doronin closed 5 days ago

dmitrii-doronin commented 2 months ago

Preserve headers order by default in CSV serialisation

Description

Current implementation does not preserve order of fields in structs. This confuses users and should be fixed.

Tasks

[ ] Implements the feature and test it thoroughly
[ ] Develop a suite of tests
[ ] Update our docs to detail the new feature
[ ] Tell what's new in changelog

NeroBlackstone commented 2 months ago

Just checked the source code, it is easy to solve if we use OrderedCollections,

I'm not sure whether to add a new dependency for this feature, but from an implementation point of view it will definitely reduce the code complexity. Although I think this can actually be solved using Vector, the readability of the code will definitely decrease. I think adding new dependencies is a better approach.

(I'm still thinking about it. You don't need to take the time to view the source code and comment, thanks!)

NeroBlackstone commented 2 months ago

I have completed a prototype implementation using OrderCollection, and obviously the order of cols actually depends on the order of to_flatten() return, however to_flatten() returns Dict, and there is no guarantee of the order in which the elements are taken out.

https://github.com/bhftbootcamp/Serde.jl/blob/85e37b4e6833c82fff8c481cbdad267b1d744e04/src/Ser/SerCsv.jl#L96-L100

So starting from this change, to_flatten() needs to return OrderedDict.

And it is impossible to save the field order in to_csv() before execute to_flatten(), we still need to flatten the structure to get column name, so we cannot bypass the call of to_flatten()