Is serialization planned, and could I contribute?

[spikespaz]

Before I grab the source and start working with it, I want to know if serialization is even considered "in scope" for this crate.

[martinohmann]

Hi @spikespaz!

I'm experimenting with serialization in the hcl-serializer branch. It's a bit out of sync with main and needs a rebase and some adjustments to recent changes.

One issue with HCL serialization is that it is highly context dependent and serializing arbitrary types does not make any sense. This is because HCL attributes and blocks both are represented as objects in JSON. It's not really possible to make this work with serde, at least not without some hacks.

My current stab at it is to require an instance of hcl::Body for serialization: https://github.com/martinohmann/hcl-rs/blob/hcl-serializer/src/ser.rs#L297

I'm still not sure if this is the right way to go which is the reason why I didn't continue working on serialization so far (and because I don't have that much time right now 😉 ).

[spikespaz]

I have seen a few solutions for this in action. There are three crates that support XML and two of them with Serde and they all have a different take.

This one uses #[serde(rename = "$value")] to get child nodes into a structure field. https://docs.rs/serde-xml-rs/0.5.1/serde_xml_rs/

And here is a similar take that uses rename again but with special prefixes like $flatten and $primitive. https://github.com/tafia/quick-xml#serde

This one is my favorite, though it isn't for Serde. I would love to see the attribute macros here worked in with serde, so we didn't have those silly special prefixes that could mess up serializing to other formats. https://lib.rs/crates/strong-xml

I think that introducing attribute macros would be the best way to go about this, but I don't really know what I'm talking about. I looked at the specification for HCL and it seems a little over-complicated for my needs. It grabbed my interest because editors have support. Before I was looking at SDLang (used by the D Programming Language community) and KDL, a derivative.

I'm writing a toy parser for Serde in my own format that is a little arbitrary, just for learning. Once I know the innerworkings of Serde better I'll come back here and see if I can save you some time. I decided that stating with contributions here may not be the best idea.

[martinohmann]

The problem with serde is that it does not support custom datatypes internally. For example, when serializing an HCL attribute or block using serde, both eventually need to be converted into a map internally. From that point on it's not possible anymore to tell if the data represents an attribute or a block.

There are workarounds for this to internally serialize these as a struct with special marker fields, but this will have the problem that these marker fields can leak out (e.g. when converting into a different encoding like JSON instead).

Here is an example from the toml crate which uses this to handle datetime fields: https://github.com/alexcrichton/toml-rs/blob/master/src/datetime.rs#L100-L107

Another workaround is to use out-of-bounds state to work around serde's limitations: https://lucumr.pocoo.org/2021/11/14/abusing-serde/

I experimented a bit with both approaches and am not really happy with them so far. I'm also not sure if attribute macros would help to fix this. I'll revisit them when I have time though.

As long as I haven't figured out how to implement serialization cleanly I'd rather not support it :wink: . As written above, the cleanest solution so far to me seems to use hcl::Body as the serialization type as it's providing all the context necessary to build the HCL representation correctly. This would be a tradeoff, but the serializer work not work with arbitrary types implementing serde::Serialize then.

[spikespaz]

Thanks for the response. Appreciate the library!

[martinohmann]

@spikespaz Just FYI: serialization support was just added via #30