feature request: <Update polars to version 0.37>

[claudiofsr]

Feature Request

Please, in the next version of rust_xlsxwriter, update polars to version 0.37.

See https://www.gitclear.com/open_repos/pola-rs/polars/release/rs-0.37.0

[jmcnamara]

I can do. However, there seems to be an issue with polars-arrow and some not very old versions of rustc:

https://github.com/pola-rs/polars/issues/14134

I may wait to see if that is fixed before upgrading. Is there any particular reason you need Polars 0.37 (just for my information)?

[claudiofsr]

New concat_str with ignore_nulls option.

[claudiofsr]

Cargo.toml:

[dependencies.polars]
version = "0.37"
features = [
    "concat_str", # Concat string data in linear time
    "lazy",       # Lazy API
    "regex",      # Use regexes in column selection
    "strings",    # Extra string utilities for Utf8Chunked
]

main.rs:

/*
fn main() {
    println!("Hello, world!");
}

#[cfg(test)]
mod tests {
    use std::error::Error;
    use polars::prelude::*;

    // cargo test -- --help
    // cargo test -- --nocapture
    // cargo test -- --show-output

    #[test]
    /**
    cargo test -- --show-output concat_str_with_null

    <https://github.com/pola-rs/polars/issues/8750>

    Add ignore_nulls for concat_str (#13877)

    geany polars-plan-0.37.0/src/dsl/functions/concat.rs&
    */
    fn concat_str_with_null() -> Result<(), Box<dyn Error>> {

        let dataframe_01: DataFrame = df!(
            "str_1" => [Some("Food"), None, Some("April"),  None],
            "str_2" => [Some("Trick"), Some("Or"), Some("Treat"),  None],
            "str_3" => [None::<&str>, None, None,  None],
            "str_4" => [Some("aa"), Some("bb"), Some("cc"),  None],
        )?;

        println!("original: {dataframe_01}\n");

        let mensagem_ignore_nulls_true: Expr = concat_str([
            col("str_1"),
            col("str_2"),
            col("str_3"),
            col("str_4"),
        ], "*", true);

        // Need add .fill_null(lit(""))
        let mensagem_ignore_nulls_false: Expr = concat_str([
            col("str_1").fill_null(lit("")),
            col("str_2").fill_null(lit("")),
            col("str_3").fill_null(lit("")),
            col("str_4").fill_null(lit("")),
        ], "*", false);

        let dataframe_02: DataFrame = dataframe_01
            .lazy()
            .with_columns([
                mensagem_ignore_nulls_true.alias("concat ignore_nulls_true"),
                mensagem_ignore_nulls_false.alias("concat ignore_nulls_false"),
            ])
            .collect()?;

        println!("dataframe02: {dataframe_02}\n");

        let col_a: Series = Series::new("concat ignore_nulls_true", &["Food*Trick*aa", "Or*bb", "April*Treat*cc", ""]);
        let col_b: Series = Series::new("concat ignore_nulls_false", &["Food*Trick**aa", "*Or**bb", "April*Treat**cc", "***"]);

        assert_eq!(dataframe_02.column("concat ignore_nulls_true")?, &col_a);
        assert_eq!(dataframe_02.column("concat ignore_nulls_false")?, &col_b);

        /*
        Output:

        original: shape: (4, 4)
        ┌───────┬───────┬───────┬───────┐
        │ str_1 ┆ str_2 ┆ str_3 ┆ str_4 │
        │ ---   ┆ ---   ┆ ---   ┆ ---   │
        │ str   ┆ str   ┆ str   ┆ str   │
        ╞═══════╪═══════╪═══════╪═══════╡
        │ Food  ┆ Trick ┆ null  ┆ aa    │
        │ null  ┆ Or    ┆ null  ┆ bb    │
        │ April ┆ Treat ┆ null  ┆ cc    │
        │ null  ┆ null  ┆ null  ┆ null  │
        └───────┴───────┴───────┴───────┘

        dataframe02: shape: (4, 6)
        ┌───────┬───────┬───────┬───────┬──────────────────────────┬───────────────────────────┐
        │ str_1 ┆ str_2 ┆ str_3 ┆ str_4 ┆ concat ignore_nulls_true ┆ concat ignore_nulls_false │
        │ ---   ┆ ---   ┆ ---   ┆ ---   ┆ ---                      ┆ ---                       │
        │ str   ┆ str   ┆ str   ┆ str   ┆ str                      ┆ str                       │
        ╞═══════╪═══════╪═══════╪═══════╪══════════════════════════╪═══════════════════════════╡
        │ Food  ┆ Trick ┆ null  ┆ aa    ┆ Food*Trick*aa            ┆ Food*Trick**aa            │
        │ null  ┆ Or    ┆ null  ┆ bb    ┆ Or*bb                    ┆ *Or**bb                   │
        │ April ┆ Treat ┆ null  ┆ cc    ┆ April*Treat*cc           ┆ April*Treat**cc           │
        │ null  ┆ null  ┆ null  ┆ null  ┆                          ┆ ***                       │
        └───────┴───────┴───────┴───────┴──────────────────────────┴───────────────────────────┘
        */

        Ok(())
    }
}

[jmcnamara]

I may wait to see if that is fixed before upgrading.

Actually, thinking about this again there isn't any need to wait for a fix. Anyone who wishes to use the Polars 0.37 with rust_xlsxwriter will already have upgraded their version of rustc.

I'll update the dependency version on main.

[jmcnamara]

Fixed on main. It should be in a release by the end of the weekend.

[jmcnamara]

This change is upstream in v0.63.0.