ClickHouse / clickhouse-rs

Official pure Rust typed client for ClickHouse DB
https://clickhouse.com
Apache License 2.0
291 stars 84 forks source link

Support for serializing maps #99

Open AdvaithD opened 8 months ago

AdvaithD commented 8 months ago

I encountered an issue with the clickhouse crate when trying to serialize a struct that uses #[serde(flatten)] to combine multiple fields into a single row. It appears that the serialize_map function is not implemented, as indicated by the todo!() macro in src/rowbinary/ser.rs at line 168, column 9.

Example:

pub struct Xyz<T: Serialize + Row> {
    #[serde(flatten)]
    pub first_field: Metadata,
    #[serde(flatten)]
    pub second_field: T,
}

From the source (src/rowbinary/ser.rs:168.9)

    #[inline]
    fn serialize_map(self, _len: Option<usize>) -> Result<Self::SerializeMap> {
        todo!();
    }

When attempting to serialize this struct, my binary panics because serialize_map is not yet supported by the crate. Is there a workaround or a planned update to support this functionality?

loyd commented 8 months ago

Hi, what result do you expect? What is the definition of the table and Metadata?

Yen commented 7 months ago

Hello, I have been trying to add metadata to rows also via #[serde(flatten)]. The patten I would expect is as follows:

#[derive(Row, Serialize)]
struct Metadata {
    foo: String,
    bar: String,
}

#[derive(Row, Serialize)]
struct MyRow {
    #[serde(flatten)]
    metadata: Metadata,
    data: String,
}

This would be expected to represent a row with columns (foo, bar, data).

I may be misunderstanding serde, but to me this seems like how the attribute should behave.

For reference I intend to use this in a templated struct to apply metadata (source/timestamp) to rows during their final submissions. But the example above is better to show the behaviour I expect.

Thanks

Edit: I realise this may be related to struct flattening, not map flattening as the issue relates, but this may be to do with terminology in serde.

elliot14A commented 1 month ago

I have similar issue but with deserialization for the following struct

#[derive(Debug, Row, Deserialize)]
pub(super) struct ClickhouseQueryMeterResult {
    // Use ClickHouse's custom serialization for datetime fields
    #[serde(with = "clickhouse::serde::time::datetime")]
    pub windowstart: OffsetDateTime,
    #[serde(with = "clickhouse::serde::time::datetime")]
    pub windowend: OffsetDateTime,
    pub value: f64,
    #[serde(flatten)]
    pub grouped_by: HashMap<String, Value>,
}

I want to deserialize all the additional fields returned from clickhouse into grouped_by field. Is there a work around for this ?