netbirdio / netbird

Connect your devices into a single secure private WireGuard®-based mesh network with SSO/MFA and simple access controls.
https://netbird.io
BSD 3-Clause "New" or "Revised" License
9.86k stars 427 forks source link

[BUG] Stucked at staring up the management container. Infinite Loop #1755

Open Luccifer opened 3 months ago

Luccifer commented 3 months ago

Description

Hello! Thanks for a great product!

I am trying to start self-hosted server. Did everything in Quick and Advanced Manual from the official site The docker-compose of management container after starting goes to infinite reboot

Steps To Reproduce

Installing on ubuntu 2204 docker.io and docker-compose from apt Walkthrough the manual Try to start -> Fail with management container

Compose Version

docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.12
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022

Docker Environment

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false

Server:
 Containers: 4
  Running: 3
  Paused: 0
  Stopped: 1
 Images: 5
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-101-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.912GiB
 Name: PQVPN-NetBird
 ID: d0590919-37e4-4af2-a215-06424bf2afa2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Logs for management container


management_1  | Error: failed to initialize database: illegal base64 data at input byte 0

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:449: loading OIDC configuration from the provided IDP configuration endpoint https://keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:454: loaded OIDC configuration from the provided IDP configuration endpoint: https://keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:456: overriding HttpConfig.AuthIssuer with a new value https://keycloak, previously configured value: https://keycloak

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:460: overriding HttpConfig.AuthKeysLocation (JWT certs) with a new value https://keycloak/protocol/openid-connect/certs, previously configured value: https://keycloak/protocol/openid-connect/certs

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:465: overriding DeviceAuthorizationFlow.TokenEndpoint with a new value: https://keycloak/protocol/openid-connect/token, previously configured value: https://keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:468: overriding DeviceAuthorizationFlow.DeviceAuthEndpoint with a new value: https://keycloak/protocol/openid-connect/auth/device, previously configured value: https://keycloak/protocol/openid-connect/auth/device

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:476: overriding DeviceAuthorizationFlow.ProviderConfig.Domain with a new value: auth.site.no, previously configured value: 

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:486: overriding PKCEAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:489: overriding PKCEAuthorizationFlow.AuthorizationEndpoint with a new value: keycloak/protocol/openid-connect/auth, previously configured value: keycloak/protocol/openid-connect/auth

management_1  | 2024-03-27T16:00:57Z INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://0.0.0.0:8081

management_1  | 2024-03-27T16:00:57Z INFO management/server/store.go:92: using SQLite store engine

management_1  | Error: failed to initialize database: illegal base64 data at input byte 0

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:449: loading OIDC configuration from the provided IDP configuration endpoint keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:454: loaded OIDC configuration from the provided IDP configuration endpoint: keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:456: overriding HttpConfig.AuthIssuer with a new value keycloak, previously configured value: keycloak

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:460: overriding HttpConfig.AuthKeysLocation (JWT certs) with a new value keycloak/protocol/openid-connect/certs, previously configured value: keycloak/protocol/openid-connect/certs

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:465: overriding DeviceAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:468: overriding DeviceAuthorizationFlow.DeviceAuthEndpoint with a new value: keycloak/protocol/openid-connect/auth/device, previously configured value: keycloak/protocol/openid-connect/auth/device

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:476: overriding DeviceAuthorizationFlow.ProviderConfig.Domain with a new value: auth.site.no, previously configured value: 

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:486: overriding PKCEAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:489: overriding PKCEAuthorizationFlow.AuthorizationEndpoint with a new value: keycloak/protocol/openid-connect/auth, previously configured value: keycloak/protocol/openid-connect/auth

management_1  | 2024-03-27T16:01:04Z INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://0.0.0.0:8081

management_1  | 2024-03-27T16:01:04Z INFO management/server/store.go:92: using SQLite store engine

management_1  | Error: failed to initialize database: illegal base64 data at input byte 0
pascal-fischer commented 3 months ago

Hi @Luccifer, the script will generate a set of config files. Can you share the content of your management.json? It looks like the DataStoreEncryptionKey is missing or wrongly created. Did you have any errors when running the script?

Luccifer commented 3 months ago

Hi @Luccifer, the script will generate a set of config files. Can you share the content of your management.json? It looks like the DataStoreEncryptionKey is missing or wrongly created. Did you have any errors when running the script?

Ofc!

Here is it from settings:

I also added quotes in this json at 49, 61, 78 and 91-92 LOC (deleted, lines at 91-92 below) 'cause ./configure.sh didnt manage to pass parsing the json, so I chacked, that thoose lines are not in quotes.. "RedirectURLs": ["$NETBIRD_AUTH_PKCE_REDIRECT_URLS"], "UseIDToken": "$NETBIRD_AUTH_PKCE_USE_ID_TOKEN"

Either with or withoud 91-92 lines still cant make working management Screenshot_1

And this one in artifacts:

{
  "Stuns": [
    {
      "Proto": "udp",
      "URI": "stun:site:3478",
      "Username": "",
      "Password": null
    }
  ],
  "TURNConfig": {
    "Turns": [
      {
        "Proto": "udp",
        "URI": "turn:site:3478",
        "Username": "self",
        "Password": "password"
      }
    ],
    "CredentialsTTL": "12h",
    "Secret": "secret",
    "TimeBasedCredentials": false
  },
  "Signal": {
    "Proto": "https",
    "URI": "site:10000",
    "Username": "",
    "Password": null
  },
  "ReverseProxy": {
    "TrustedHTTPProxies": [],
    "TrustedHTTPProxiesCount": 0,
    "TrustedPeers": [
      "0.0.0.0/0"
    ]
  },
  "Datadir": "",
  "DataStoreEncryptionKey": "someplaintextpassword",
  "StoreConfig": {
    "Engine": "sqlite"
  },
  "HttpConfig": {
    "Address": "0.0.0.0:33073",
    "AuthIssuer": "https://keycloak.site",
    "AuthAudience": "Client5",
    "AuthKeysLocation": "https://keycloak.site/protocol/openid-connect/certs",
    "AuthUserIDClaim": "",
    "CertFile": "",
    "CertKey": "",
    "IdpSignKeyRefreshEnabled": true,
    "OIDCConfigEndpoint": "https://keycloak.site/.well-known/openid-configuration"
  },
  "IdpManagerConfig": {
    "ManagerType": "keycloak",
    "ClientConfig": {
      "Issuer": "https://keycloak.site",
      "TokenEndpoint": "https://keycloak.site/protocol/openid-connect/token",
      "ClientID": "netbird-backend",
      "ClientSecret": "SuperClientSecretInPlainText",
      "GrantType": "client_credentials"
    },
    "ExtraConfig": {
      "AdminEndpoint": "https://keycloak.site"
    },
    "Auth0ClientCredentials": null,
    "AzureClientCredentials": null,
    "KeycloakClientCredentials": null,
    "ZitadelClientCredentials": null
  },
  "DeviceAuthorizationFlow": {
    "Provider": "hosted",
    "ProviderConfig": {
      "Audience": "Client5",
      "AuthorizationEndpoint": "",
      "Domain": "",
      "ClientID": "Client5",
      "ClientSecret": "",
      "TokenEndpoint": "https://keycloak.site/protocol/openid-connect/token",
      "DeviceAuthEndpoint": "https://keycloak.site/protocol/openid-connect/auth/device",
      "Scope": "openid",
      "UseIDToken": false,
      "RedirectURLs": null
    }
  },
  "PKCEAuthorizationFlow": {
    "ProviderConfig": {
      "Audience": "Client5",
      "ClientID": "Client5",
      "ClientSecret": "",
      "Domain": "",
      "AuthorizationEndpoint": "https://keycloak.site/protocol/openid-connect/auth",
      "TokenEndpoint": "https://keycloak.site/protocol/openid-connect/token",
      "Scope": "openid profile email offline_access api",
      "RedirectURLs": [
        "http://localhost:53000"
      ],
      "UseIDToken": false
    }
  }
}

If you mean the ./configure.sh script, after I made changes in manifest.json no errors were outputed only notices:

Letsencrypt was disabled, the Https-endpoints cannot be used anymore
 and a reverse-proxy with Https needs to be placed in front of netbird!
The following forwards have to be setup:
- https://site.no:443 -http-> dashboard:80
- https://site.no:33073/api -http-> management:33073
- https://site.no:33073/management.ManagementService/ -grpc-> management:33073
- https://site.no:10000/signalexchange.SignalExchange/ -grpc-> signal:80
You most likely also have to change NETBIRD_MGMT_API_ENDPOINT in base.setup.env and port-mappings in docker-compose.yml.tmpl and rerun this script.
 The target of the forwards depends on your setup. Beware of the gRPC protocol instead of http for management and signal!
You are also free to remove any occurrences of the Letsencrypt-volume netbird-letsencrypt

Anyways, i didnt find any occurances of NETBIRD_DATASTORE_ENC_KEY in configs..

Luccifer commented 3 months ago

Did enyone managed to make it start from scratch with advanced tutorial with latest tag, assuming management.json has issues in json-code + some mappings are abscent

Luccifer commented 3 months ago

Well after 5th clean installation VM Ubuntu 2204 proxmox I made it working with standalone docker compose. Also I spotted that ./configure.sh sometimes creates folder of management.json/ instead of management.json