knadh / koanf

Simple, extremely lightweight, extensible, configuration management library for Go. Support for JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper.
MIT License
2.56k stars 146 forks source link

Support for merging slices #269

Closed SamerJ closed 2 weeks ago

SamerJ commented 5 months ago

As of today, the library works in most scenarios, but it has one annoying downside; it doesn't support slices. At least not in the way you'd expect like merging them. The library does support adding a custom merge function, but it's not scoped and you end up writing the entire merge algorithm. Another library I came across had the ability to define merge functions for certain types. In which case, you could provide a merge function for Slices and rely on the default behavior for everything else.

Any chance this library can be improved by adding scoped merge functions? Are there any workarounds that can be employed to end up merging slices?

knadh commented 5 months ago

Can you give an example with some pseudocode illustrating what you're referring to?

SamerJ commented 5 months ago

@knadh

Take this struct for example:

type Parent struct {
    Name  string `koanf:"name"`
    Child Child  `koanf:"child"`
}

type Child struct {
    Item1 map[string][]string `koanf:"item1"`
}

Assume you have 2 "Parent" structs you would like to merge.

    parent1 := Parent{
        Name: "name",
        Child: Child{
            Item1: map[string][]string{
                "Key1": {"value1", "value2"},
            },
        },
    }

    parent2 := Parent{
        Name: "new name",
        Child: Child{
            Item1: map[string][]string{
                "Key1": {"value3"},
                "Key2": {"value1"},
            },
        },
    }

Intended result:

    mergedParent := Parent{
        Name: "new name",
        Child: Child{
            Item1: map[string][]string{
                "Key1": {"value1", "value2", "value3"},
                "Key2": {"value1"},
            },
        },
    }

However, as is known, koanf will simply replace slice values rather than append them. Actual result:

    actualMergedParent := Parent{
        Name: "new name",
        Child: Child{
            Item1: map[string][]string{
                "Key1": {"value3"},
                "Key2": {"value1"},
            },
        },
    }

The suggestion/request is to have something like

k.Load(structs.Provider(config, "koanf"), nil, koanf.WithMergeFuncForType(reflect.Slice, mergeFunc)) 

As a result, you would only need to write the code to handle slices and rely on the default merging algorithm for anything else. WDYT?

Inspired from another library I had seen: https://github.com/InVisionApp/conjungo

    opts := conjungo.NewOptions()
    opts.SetKindMergeFunc(reflect.Slice, func(target, source reflect.Value, o *conjungo.Options) (reflect.Value, error) {
        //...
        return target, nil
    })
    err = conjungo.Merge(dest, src, opts)
knadh commented 2 weeks ago

Closing this in favour of: https://github.com/knadh/koanf/issues/299#issuecomment-2199931008