mehcode / config-rs

⚙️ Layered configuration system for Rust applications (with strong support for 12-factor applications).
Apache License 2.0
2.43k stars 206 forks source link

failed type conversion in .ini file for serde flatten #504

Closed mfirhas closed 6 days ago

mfirhas commented 6 months ago

I think other formats are all fine, but for .ini/.env, when I flatten the structs into another struct, type conversion failed. I checked the code and found this:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/file/format/ini.rs#L21 https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/file/format/ini.rs#L30

All code are string-ed rightaway.

Should it be parsed like this?: https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/env.rs#L282

I test locally and it works and was about to create PR, but confuse with master branch status I faced unexpected error, unlike in v0.13.4.

matthiasbeyer commented 6 months ago

Hi, thanks for reporting this! A PR would be awesome of course!

, but confuse with master branch status I faced unexpected error,

What do you mean by this?

mfirhas commented 6 months ago

I created my own test code, test it on master branch got error missing field 'field_name', switched to v0.13.4 got no error. The test was for loading configs from .env/.ini file.

matthiasbeyer commented 6 months ago

The last CI run on master is green though.

mfirhas commented 6 months ago

Here is my test code:

// tests/file_env_test.rs

use config::{
    builder::DefaultState, Config, ConfigBuilder, ConfigError, Environment, File, FileFormat,
};
use serde::Deserialize;
use std::fmt::{Debug, Display};

const ALL_CFG_FILE: &str = "./tests/all.env";

#[derive(Debug, Deserialize)]
struct TestConfig {
    #[serde(alias = "TEST__NUMBER")]
    pub number: i32,
    #[serde(alias = "TEST__STRING")]
    pub string: String,
    #[serde(alias = "TEST__FLOAT")]
    pub float: f32,
    #[serde(alias = "TEST__BOOLEAN")]
    pub boolean: bool,
}

#[test]
fn test_file_config_success() {
    let cfg = Config::builder()
        .add_source(config::File::new(ALL_CFG_FILE, FileFormat::Ini))
        .build()
        .unwrap()
        .try_deserialize::<TestConfig>();
    dbg!(&cfg);
    assert!(cfg.is_ok());
}

ini file tests/all.env

TEST__NUMBER=123
TEST__STRING=this is string from file
TEST__FLOAT=123.3
TEST__BOOLEAN=true

My original issue is about type incompatibility on nested configs parsing from .ini file. Above test code is only to test between master and v0.13.4, but found said error(missing field) on master.

mfirhas commented 6 months ago

There're many commits unmerged from v0.13.4 into master, hence different version.

mfirhas commented 6 months ago

This is the my commit id: e30debf7cab0b4b8e0edbd254cdc58edc24ab7b2

polarathene commented 6 months ago

All code are string-ed rightaway.

For reference the INI support is being refactored a little here: https://github.com/mehcode/config-rs/pull/470

That doesn't change the logic you draw attention to though. The reason we are calling .to_owned() there is because rust-ini is used and it provides an iterator over &str, there is no type information available. To "import" that data we need to create a copy of the strings it provides.

The other formats often parse the strings themselves and provide a Value enum that can then be mapped into the equivalent for config-rs (which normalizes Value type for internal use).

Should it be parsed like this?:

Yes, that sort of improvement could probably be added to (or after) the referenced refactor PR. It is intended for 0.14 so as a breaking change would be ok AFAIK, @matthiasbeyer ?

It can be a bit problematic depending on the value I think 🤷‍♂️


Here a string to a float:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/value.rs#L521-L540

and likewise string (or whatever other intermediary value in our container) to a bool type of the user config struct:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/value.rs#L233-L245

I think the issue is more related to that area, than rust-ini and if you attempt to fix it with the ini format you risk introducing regressions and less flexibility 😅

mfirhas commented 6 months ago

Okay, then I'll just need to use that version after release.

polarathene commented 6 months ago

when I flatten the structs into another struct, type conversion failed

I believe there are other issues related to serde deserializing (the internal value.rs representation, into user struct) within the project?


I'll just need to use that version after release.

I don't know if your concern will be fixed. Addressing INI to parse from &str to a ValueKind may fix your concern, but replace it with a regression that impacts others, which is not really a fix.

The value should already be captured into Value (value.rs) as a String and from there to the desired type. The fix you want should be handled there instead, but as shown above there are some cases where serde or the internal usage of it within config-rs is not compatible with some serde attributes like #[serde(flatten)].

Some formats are also known to be lossy in conversion into the config-rs Value container.

mfirhas commented 6 months ago

For now, I'll just refer to my commit in my fork. After this is fixed/refactored as a whole, will switch back.