Upgrade YAML fails after version `0.107.37`

dnomd343 commented 9 months ago

Prerequisites

[X] I have checked the Wiki and Discussions and found no answer
[X] I have searched other issues and found no duplicates
[X] I want to report a bug and not ask a question or ask for help
[X] I have set up AdGuard Home correctly and configured clients to use it. (Use the Discussions for help with installing and configuring clients.)

Platform (OS and CPU architecture)

Linux, AMD64 (aka x86_64)

Installation

Custom package (OpenWrt, HomeAssistant, etc; please mention in the description)

Setup

On one machine

AdGuard Home version

0.107.37

Action

Use a basic JSON configuration file in /tmp/AdGuardHome.yaml.

YAML is a superset of JSON, and all legal JSON configuration files can be parsed by YAML.

The Bcrypt key here is not really used.

{
  "users": [
    {
      "name": "admin",
      "password": "$2a$10$q0t.WVsqQ7yo5NRMX2Rareim2BrzEnbPGzuLh9CxdFA6G9BNsncN6"
    }
  ],
  "dns": {
    "port": 53,
    "bind_host": "0.0.0.0",
    "upstream_dns": [
      "127.0.0.1:5353"
    ],
    "upstream_dns_file": "",
    "bootstrap_dns": []
  }
}

In version 0.107.36 I can get it to boot and upgrade to the latest YAML configuration.

> AdGuardHome-0.107.36 --work-dir /tmp/
2024/01/27 06:04:27.091423 [info] AdGuard Home, version v0.107.36
2024/01/27 06:04:27.091566 [info] home.upgradeSchema0to1(): called
2024/01/27 06:04:27.091575 [info] deleting /tmp/dnsfilter.txt as we don't need it anymore
2024/01/27 06:04:27.091592 [info] home.upgradeSchema1to2(): called
2024/01/27 06:04:27.091598 [info] deleting /tmp/Corefile as we don't need it anymore
2024/01/27 06:04:27.091613 [info] home.upgradeSchema2to3(): called
2024/01/27 06:04:27.091623 [info] home.upgradeSchema3to4(): called
2024/01/27 06:04:27.091635 [info] home.upgradeSchema4to5(): called
2024/01/27 06:04:27.091638 [info] Upgrade yaml: 5 to 6
2024/01/27 06:04:27.091640 [info] Upgrade yaml: 6 to 7
2024/01/27 06:04:27.091659 [info] Upgrade yaml: 7 to 8
2024/01/27 06:04:27.091665 [info] Upgrade yaml: 8 to 9
2024/01/27 06:04:27.091668 [info] Upgrade yaml: 9 to 10
2024/01/27 06:04:27.091673 [info] Upgrade yaml: 10 to 11
2024/01/27 06:04:27.091676 [info] Upgrade yaml: 11 to 12
2024/01/27 06:04:27.091681 [info] Upgrade yaml: 12 to 13
2024/01/27 06:04:27.091685 [info] Upgrade yaml: 13 to 14
2024/01/27 06:04:27.091689 [info] Upgrade yaml: 14 to 15
2024/01/27 06:04:27.091695 [info] Upgrade yaml: 15 to 16
2024/01/27 06:04:27.091702 [info] Upgrade yaml: 16 to 17
2024/01/27 06:04:27.091706 [info] Upgrade yaml: 17 to 18
2024/01/27 06:04:27.091720 [info] Upgrade yaml: 18 to 19
2024/01/27 06:04:27.091727 [info] Upgrade yaml: 19 to 20
2024/01/27 06:04:27.091730 [info] Upgrade yaml: 20 to 21
2024/01/27 06:04:27.091733 [info] Upgrade yaml: 21 to 22
2024/01/27 06:04:27.091746 [info] Upgrade yaml: 22 to 23
2024/01/27 06:04:27.091754 [info] Upgrade yaml: 23 to 24
2024/01/27 06:04:27.094172 [info] tls: using default ciphers
2024/01/27 06:04:27.097802 [info] safesearch default: reset 253 rules
2024/01/27 06:04:27.100963 [info] Initializing auth module: /tmp/data/sessions.db
2024/01/27 06:04:27.102499 [info] auth: initialized.  users:1  sessions:0
2024/01/27 06:04:27.102513 [info] web: initializing
2024/01/27 06:04:27.104547 [info] dnsproxy: cache: enabled, size 4096 b
2024/01/27 06:04:27.104557 [info] dnsproxy: max goroutines is set to 300
2024/01/27 06:04:27.104930 [info] AdGuard Home is available at the following addresses:
2024/01/27 06:04:27.105055 [info] go to http://127.0.0.1:3000
2024/01/27 06:04:27.105059 [info] go to http://172.18.0.5:3000
2024/01/27 06:04:27.105272 [info] clients: processing addresses
2024/01/27 06:04:27.111258 [info] dnsproxy: starting dns proxy server
2024/01/27 06:04:27.111300 [info] Ratelimit is enabled and set to 20 rps
2024/01/27 06:04:27.111310 [info] The server is configured to refuse ANY requests
2024/01/27 06:04:27.111315 [info] dnsproxy: cache: enabled, size 4194304 b
2024/01/27 06:04:27.111322 [info] dnsproxy: max goroutines is set to 300
2024/01/27 06:04:27.111339 [info] dnsproxy: creating udp server socket 0.0.0.0:53
2024/01/27 06:04:27.111429 [info] dnsproxy: listening to udp://[::]:53
2024/01/27 06:04:27.111436 [info] dnsproxy: creating tcp server socket 0.0.0.0:53
2024/01/27 06:04:27.111492 [info] dnsproxy: listening to tcp://[::]:53
2024/01/27 06:04:27.111827 [info] dnsproxy: entering udp listener loop on [::]:53
2024/01/27 06:04:27.111921 [info] dnsproxy: entering tcp listener loop on [::]:53

But in the 0.107.37 version, an error occurred and the configuration file was upgraded to the wrong YAML.

> AdGuardHome-0.107.37 --work-dir /tmp/
2024/01/27 06:03:11.032715 [info] AdGuard Home, version v0.107.37
2024/01/27 06:03:11.032895 [info] Upgrade yaml: 0 to 1
2024/01/27 06:03:11.032906 [info] deleting /tmp/dnsfilter.txt as we don't need it anymore
2024/01/27 06:03:11.032920 [info] Upgrade yaml: 1 to 2
2024/01/27 06:03:11.032926 [info] deleting /tmp/Corefile as we don't need it anymore
2024/01/27 06:03:11.032938 [info] Upgrade yaml: 2 to 3
2024/01/27 06:03:11.032945 [info] Upgrade yaml: 3 to 4
2024/01/27 06:03:11.032947 [info] Upgrade yaml: 4 to 5
2024/01/27 06:03:11.032952 [info] Upgrade yaml: 5 to 6
2024/01/27 06:03:11.032955 [info] Upgrade yaml: 6 to 7
2024/01/27 06:03:11.032968 [info] Upgrade yaml: 7 to 8
2024/01/27 06:03:11.032972 [info] Upgrade yaml: 8 to 9
2024/01/27 06:03:11.032976 [info] Upgrade yaml: 9 to 10
2024/01/27 06:03:11.032981 [info] Upgrade yaml: 10 to 11
2024/01/27 06:03:11.032987 [info] Upgrade yaml: 11 to 12
2024/01/27 06:03:11.032995 [info] Upgrade yaml: 12 to 13
2024/01/27 06:03:11.033007 [info] Upgrade yaml: 13 to 14
2024/01/27 06:03:11.033011 [info] Upgrade yaml: 14 to 15
2024/01/27 06:03:11.033021 [info] Upgrade yaml: 15 to 16
2024/01/27 06:03:11.033024 [info] Upgrade yaml: 16 to 17
2024/01/27 06:03:11.033029 [info] Upgrade yaml: 17 to 18
2024/01/27 06:03:11.033038 [info] Upgrade yaml: 18 to 19
2024/01/27 06:03:11.033041 [info] Upgrade yaml: 19 to 20
2024/01/27 06:03:11.033045 [info] Upgrade yaml: 20 to 21
2024/01/27 06:03:11.033054 [info] Upgrade yaml: 21 to 22
2024/01/27 06:03:11.033057 [info] Upgrade yaml: 22 to 23
2024/01/27 06:03:11.033060 [info] Upgrade yaml: 23 to 24
2024/01/27 06:03:11.033064 [info] Upgrade yaml: 24 to 25
2024/01/27 06:03:11.033068 [info] Upgrade yaml: 25 to 26
2024/01/27 06:03:11.033081 [info] Upgrade yaml: 26 to 27
2024/01/27 06:03:11.035575 [error] parsing configuration file: yaml: unmarshal errors:
  line 13: cannot unmarshal !!seq into string

At this time, the output configuration/tmp/AdGuardHome.yaml is as follows:

···
dns:
  bind_hosts:
    - 0.0.0.0
  bootstrap_dns:
    - []
  edns_client_subnet:
    custom_ip: ""
    enabled: false
    use_custom: false
  port: 53
  upstream_dns:
    - 127.0.0.1:5353
  upstream_dns_file: ""
···

It can be found that an error occurred in bootstrap_dns . The [] of JSON (or YAML) was treated as a string, which is an illegal YAML syntax.

Expected result

YAML is upgraded correctly, like:

···
dns:
  bind_hosts:
    - 0.0.0.0
  bootstrap_dns: []
  edns_client_subnet:
    custom_ip: ""
    enabled: false
    use_custom: false
  port: 53
  upstream_dns:
    - 127.0.0.1:5353
  upstream_dns_file: ""
···

Actual result

A valid JSON (or YAML) input was upgraded to illegal markup.

Additional information and/or screenshots

It has nothing to do with the system and exists on all hosts.

dnomd343 commented 9 months ago

I retried the YAML structure, and it still gave an error. The bootstrap_dns was incorrectly nested with one more slice.

Input configure file:

users:
  - name: admin
    password: $2a$10$q0t.WVsqQ7yo5NRMX2Rareim2BrzEnbPGzuLh9CxdFA6G9BNsncN6
dns:
  port: 53
  bind_host: 0.0.0.0
  upstream_dns:
    - '127.0.0.1:5353'
  upstream_dns_file: ''
  bootstrap_dns:
    - 8.8.8.8

Upgraded configure file:

···
dns:
  bind_hosts:
    - 0.0.0.0
  bootstrap_dns:
    - - 8.8.8.8
  edns_client_subnet:
    custom_ip: ""
    enabled: false
    use_custom: false
  port: 53
  upstream_dns:
    - 127.0.0.1:5353
  upstream_dns_file: ""
···

- - 8.8.8.8 is the result of nesting twice. I tend to have a BUG in the upgrade of 25, 26 or 27.

ainar-g commented 9 months ago

This isn't a typical way to get a configuration file, see/upvote #4199 for a better solution in the future. As for the configuration, your initial bootstrap_dns should be a string or probably just absent. See configuration migrator's test data for examples.

dnomd343 commented 9 months ago

Thanks for the reply, I was able to trick it into not initiating a YAML upgrade by specifying the schema_version and instead initializing it with default parameters.

For example:

{
  "users": [
    {
      "name": "admin",
      "password": "$2a$10$q0t.WVsqQ7yo5NRMX2Rareim2BrzEnbPGzuLh9CxdFA6G9BNsncN6"
    }
  ],
  "dns": {
    "port": 53,
    "bind_host": "0.0.0.0",
    "upstream_dns": [
      "127.0.0.1:5353"
    ],
    "upstream_dns_file": "",
    "bootstrap_dns": []
  },
  "schema_version": 27
}

AdguardTeam / AdGuardHome