Inner map on structs are not considered when unmarshalling with viper_bind_struct tag

lucasoares commented 3 months ago

Preflight Checklist

[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.
[X] I am not looking for support or already pursued the available support channels without success.
[X] I have checked the troubleshooting guide for my problem, without success.

Viper Version

1.12.2

Go Version

1.22.1

Config Source

Environment variables, Files

Format

JSON

Repl.it link

No response

Code reproducing the issue

package config

import (
    "bytes"
    "os"
    "strings"
    "testing"

    "github.com/mitchellh/mapstructure"
    "github.com/spf13/viper"
    "github.com/stretchr/testify/require"
)

type MapData struct {
    Enabled bool `json:"enabled" mapstructure:"enabled"`
}

type Config struct {
    MapData map[string]MapData `json:"map_data" mapstructure:"map_data"`
}

var jsonData = []byte(`{
    "map_data": {
        "first": {
            "enabled": false
        },
        "second": {
            "enabled": true
        }
    }
}`)

func TestSetup(t *testing.T) {
    os.Setenv("EXAMPLE_MAP_DATA_FIRST_ENABLED", "true")
    defer os.Unsetenv("EXAMPLE_MAP_DATA_FIRST_ENABLED")

    os.Setenv("EXAMPLE_MAP_DATA_SECOND_ENABLED", "false")
    defer os.Unsetenv("EXAMPLE_MAP_DATA_SECOND_ENABLED")

    viper.SetConfigType("json")
    viper.ReadConfig(bytes.NewBuffer(jsonData))

    viper.SetTypeByDefaultValue(true)
    viper.SetEnvPrefix("example")
    viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
    viper.AutomaticEnv()

    var data Config
    err := viper.Unmarshal(&data, viper.DecodeHook(
        mapstructure.ComposeDecodeHookFunc(
            mapstructure.TextUnmarshallerHookFunc(),
            mapstructure.StringToTimeDurationHookFunc(),
            mapstructure.StringToSliceHookFunc(","),
        ),
    ))

    require.NoError(t, err)
    require.NotNil(t, data.MapData["first"])  // This will not fail, it is getting from the json file
    require.NotNil(t, data.MapData["second"]) // This will not fail, it is getting from the json file

    require.True(t, data.MapData["first"].Enabled)   // This will fail
    require.False(t, data.MapData["second"].Enabled) // This will fail
}

Expected Behavior

It should be possible to bind map values through environment variables.

Actual Behavior

The environment variable is not considered.

Steps To Reproduce

Execute the code I provided.

Additional Information

When coding this, I thought it would be hard for Viper to consider this complex structure. How will the code know the map's key from the environment variable? Maybe it should have a specific separator for map keys? Like EXAMPLE_MAP_DATA__FIRST__ENABLED instead of EXAMPLE_MAP_DATA_FIRST_ENABLED? I don't know if I'm doing something wrong here.

I didn't manage to create a new map key from an environment variable. I think the documentation for environment variables when using maps, slices, etc is kind of outdated and must be updated. I'm doing trial-error for several minutes trying to make this work and I don't have any idea if I'm in the right direction.

github-actions[bot] commented 3 months ago

👋 Thanks for reporting!

A maintainer will take a look at your issue shortly. 👀

In the meantime: We are working on Viper v2 and we would love to hear your thoughts about what you like or don't like about Viper, so we can improve or fix those issues.

⏰ If you have a couple minutes, please take some time and share your thoughts: https://forms.gle/R6faU74qPRPAzchZ9

📣 If you've already given us your feedback, you can still help by spreading the news, either by sharing the above link or telling people about this on Twitter:

https://twitter.com/sagikazarmark/status/1306904078967074816

Thank you! ❤️

sagikazarmark commented 1 month ago

1.12.2

Is this really the Viper version you are using? It's pretty old.

I just ran your test locally, and it passed.

The reason for that is you read the JSON configuration first, which will populate Viper with those keys. Bind struct is not in play here.

If you remove the ReadConfig part, the test does fail, but there is an explanation: mapstructure can't distinguish between maps and structs which means any keys missing from the encoded version will also be missing from automatic env (as you point out in your analysis).

I'm not quite sure what the right solution is in this case. Maybe some sort of glob matching to allow manually bind env vars without having to specify the map keys?

In any case, the above test should work if you have those values defined in a config file.

spf13 / viper