search

knadh / koanf

Simple, extremely lightweight, extensible, configuration management library for Go. Support for JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper.
MIT License
2.53k stars 145 forks source link

koanf reports empty string key after a file gets modified #305

Open matheusjunior opened 4 days ago

matheusjunior commented 4 days ago

Describe the bug File watcher prints empty string data after file gets modified a couple of times.

To Reproduce

  1. Run the go snippet below
  2. Modify rpc value from config.yaml file
  3. Save config.yaml
  4. Repeat steps 2-3 until issue happens

main.go:

package main

import (
    "fmt"
    "log"
    "time"

    "github.com/knadh/koanf/parsers/yaml"
    "github.com/knadh/koanf/providers/file"
    "github.com/knadh/koanf/v2"
)

func main() {
    k := koanf.New(".")
    provider := file.Provider("config.yaml")
    err := k.Load(provider, yaml.Parser())
    if err != nil {
        panic(err)
    }

    provider.Watch(func(event interface{}, err error) {
        if err != nil {
            log.Printf("watch error: %v", err)
            return
        }

        // Throw away the old config and load a fresh copy.
        log.Println("config changed. Reloading ...")
        k = koanf.New(".")
        err = k.Load(provider, yaml.Parser())
        if err != nil {
            log.Printf("watch error: %v", err)
            return
        }
    })

    go func() {
        for {
            fmt.Println(k.String("rpc"))
            time.Sleep(time.Second)
        }
    }()

    <-make(chan bool)
}

config.yaml:

rpc: "ddf"

Screenshot showing the described behavior: Screenshot from 2024-07-03 18-53-43

Expected behavior Updated rpc value is correctly shown on logs.

Please provide the following information):

Additional context

rhnvrm commented 4 days ago

Running the sample code with go run -race main.go suggests that there are data races

go run -race main.go 
xddf
xddf
xddf
2024/07/04 11:32:10 config changed. Reloading ...
==================
WARNING: DATA RACE
Read at 0x00c000112040 by goroutine 9:
  main.main.func2()
      /tmp/test/main.go:39 +0x2e

Previous write at 0x00c000112040 by goroutine 8:
  main.main.func1()
      /tmp/test/main.go:29 +0x38e
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a0d0 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:318 +0x65
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a0d0 by goroutine 8:
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:420 +0x279
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a240 by goroutine 9:
  runtime.mapaccess2_faststr()
      /usr/local/go/src/runtime/map_faststr.go:108 +0x0
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:318 +0x8d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a240 by goroutine 8:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:203 +0x0
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:509 +0x5e7
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c0001c21e8 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:318 +0x9e
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c0001c21e8 by goroutine 8:
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:509 +0x5f6
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a0c0 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:322 +0xee
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a0c0 by goroutine 8:
  github.com/knadh/koanf/v2.NewWithConf()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:80 +0x28e
  github.com/knadh/koanf/v2.New()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:71 +0x254
  main.main.func1()
      /tmp/test/main.go:29 +0x382
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c000016130 by goroutine 9:
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/maps@v0.1.1/maps.go:216 +0x47
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c000016130 by goroutine 8:
  runtime.slicecopy()
      /usr/local/go/src/runtime/slice.go:325 +0x0
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:510 +0x694
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a030 by goroutine 9:
  runtime.mapaccess2_faststr()
      /usr/local/go/src/runtime/map_faststr.go:108 +0x0
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/maps@v0.1.1/maps.go:216 +0x67
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a030 by goroutine 8:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:203 +0x0
  github.com/knadh/koanf/maps.Merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/maps@v0.1.1/maps.go:114 +0x253
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:416 +0x13c
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c0001b42c8 by goroutine 9:
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/maps@v0.1.1/maps.go:216 +0x75
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c0001b42c8 by goroutine 8:
  github.com/knadh/koanf/maps.Merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/maps@v0.1.1/maps.go:114 +0x264
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:416 +0x13c
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/v2@v2.1.1/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/file@v1.0.0/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================