env prefix and env key replacer not applied to dotenv file

xobotyi commented 3 months ago

Preflight Checklist

[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.
[X] I am not looking for support or already pursued the available support channels without success.
[X] I have checked the troubleshooting guide for my problem, without success.

Viper Version

1.18.2

Go Version

1.22

Config Source

Files

Format

Dotenv

Repl.it link

No response

Code reproducing the issue

package main

import (
    "fmt"
    "gaijin/lib/must"
    "github.com/spf13/viper"
    "strings"
)

type BarConfig struct {
    Baz int
}

type Config struct {
    Foo int

    Bar BarConfig `mapstructure:"baar"`
}

func main() {
    v := viper.New()
    v.SetEnvPrefix("PREFIX")
    v.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
    v.AutomaticEnv()
    v.SetConfigFile("./.config.env")

    _ = v.ReadInConfig()

    var cfg Config
    must.NoErr(v.Unmarshal(&cfg))

    fmt.Printf("%#v\n", cfg)
}

Expected Behavior

Dotenv files expected to be threated exactly as env variables

Actual Behavior

Neither env prefix, nor env key replacer not applied to configs being read from dotenv file

Steps To Reproduce

No response

Additional Information

example dotenv file content

PREFIX_FOO=123
PREFIX_BAAR_BAZ=256

github-actions[bot] commented 3 months ago

👋 Thanks for reporting!

A maintainer will take a look at your issue shortly. 👀

In the meantime: We are working on Viper v2 and we would love to hear your thoughts about what you like or don't like about Viper, so we can improve or fix those issues.

⏰ If you have a couple minutes, please take some time and share your thoughts: https://forms.gle/R6faU74qPRPAzchZ9

📣 If you've already given us your feedback, you can still help by spreading the news, either by sharing the above link or telling people about this on Twitter:

https://twitter.com/sagikazarmark/status/1306904078967074816

Thank you! ❤️

FaheemuddinKhan commented 3 months ago

Thank you for raising this issue!

Upon investigation, it seems that environment variable prefixes and key replacers are not applied to configurations read from a dotenv file.

Explanation

The merging with prefixes and key replacers is only applied when AutomaticEnv is used. AutomaticEnv monitors the OS environment variables, and on any subsequent call to viper.Get(), it checks these environment variables, applying the merge and replacer functions as specified.

In contrast, when using ReadInConfig() to load a dotenv file, Viper uses the in-built encoding Decoder. The configurations are read directly from the file, without applying the environment prefixes or key replacers. This is because ReadInConfig()focuses on loading and decoding the file contents as-is, without integrating the environment-related settings from AutomaticEnv.

Conclusion

To summarize, environment variables are required if you need to apply environment prefixes and key replacers, as these are not automatically applied to configurations loaded from a dotenv file using ReadInConfig().

xobotyi commented 3 months ago

I understand, I've read the sources. Main issue here that structures nesting, along with config format interopability simply won't work. Mapsrtucture uses . as nesting separator which, obviously won't happen within dotenv file. Same for yaml-dotenv interop in such case switching format requires rewriting all configs (mapsrtucture tags), as yaml conventionally uses dash as words separator.

What I would expect, as said, dotenv file would be 1 to 1 compatible system envs replacement. Which is not the case now.

What I would suggest is to mirror automatic envs methods with ones that will be applied to dotenv files. Such approach will bring functionality to those who need it. Without breaking backwards compatibility.

sagikazarmark commented 2 months ago

Dotenv files expected to be threated exactly as env variables

That's certainly an interesting proposition. While I see what that would make sense, architecturally, dotenv files are processed very differently from actual environment variables.

One thing you could do as a workaround (starting 1.20) is wrapping the dotenv codec and implementing key replacement yourself: https://github.com/spf13/viper/blob/master/encoding.go

I would certainly consider adding it as an option here as well.

In any case, the above workaround should work in the short term.

spf13 / viper