search

temporalio / helm-charts

Temporal Helm charts
MIT License
305 stars 337 forks source link

Invalid archival config #437

Closed Nepoxx closed 11 months ago

Nepoxx commented 11 months ago

Version 1.22.1

When setting up archival using a config very similar to https://github.com/temporalio/helm-charts/blob/master/values/values.archival.s3.yaml I get:

TEMPORAL_ADDRESS is not set, setting it to 10.32.2.46:7233
2023/10/26 18:10:35 Loading config; env=docker,zone=,configDir=config
2023/10/26 18:10:35 Loading config files=[config/docker.yaml]
{"level":"info","ts":"2023-10-26T18:10:35.038Z","msg":"Build info.","git-time":"2023-09-05T21:15:51.000Z","git-revision":"7e4bacdecf97216338e9a859fbccd42a727cbe68","git-modified":true,"go-arch":"amd64","go-os":"linux","go-version":"go1.20.6","cgo-enabled":false,"server-version":"1.22.0","debug-mode":false,"logging-call-at":"main.go:148"}
{"level":"info","ts":"2023-10-26T18:10:35.039Z","msg":"Updated dynamic config","logging-call-at":"file_based_client.go:195"}
{"level":"warn","ts":"2023-10-26T18:10:35.039Z","msg":"Not using any authorizer and flag `--allow-no-auth` not detected. Future versions will require using the flag `--allow-no-auth` if you do not want to set an authorizer.","logging-call-at":"main.go:178"}
[Fx] PROVIDE    *pprof.PProfInitializerImpl <= go.temporal.io/server/common/pprof.NewInitializer()
[Fx] PROVIDE    *temporal.ServerImpl <= go.temporal.io/server/temporal.NewServerFxImpl()
[Fx] PROVIDE    *temporal.serverOptions <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    chan interface {} <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    temporal.synchronizationModeParams <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    *config.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    *config.PProf <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    log.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    resource.ServiceNames <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    resource.NamespaceLogger <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    resolver.ServiceResolver <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    client.AbstractDataStoreFactory <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    searchattribute.Mapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    []grpc.UnaryServerInterceptor <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    authorization.Authorizer <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    authorization.ClaimMapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    authorization.JWTAudienceMapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    log.Logger <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    client.FactoryProvider <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    dynamicconfig.Client <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    encryption.TLSConfigProvider <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    *client.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    client.Client <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    metrics.Handler <= go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] PROVIDE    []trace.SpanExporter <= go.temporal.io/server/temporal.glob..func2()
[Fx] PROVIDE    client.FactoryProviderFn <= go.temporal.io/server/temporal.PersistenceFactoryProvider()
[Fx] PROVIDE    *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.HistoryServiceProvider()
[Fx] PROVIDE    *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.MatchingServiceProvider()
[Fx] PROVIDE    *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.FrontendServiceProvider()
[Fx] PROVIDE    *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.InternalFrontendServiceProvider()
[Fx] PROVIDE    *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.WorkerServiceProvider()
[Fx] PROVIDE    *cluster.Config <= go.temporal.io/server/temporal.ApplyClusterMetadataConfigProvider()
[Fx] PROVIDE    config.Persistence <= go.temporal.io/server/temporal.ApplyClusterMetadataConfigProvider()
[Fx] SUPPLY    []temporal.ServerOption
[Fx] PROVIDE    fx.Lifecycle <= go.uber.org/fx.New.func1()
[Fx] PROVIDE    fx.Shutdowner <= go.uber.org/fx.(*App).shutdowner-fm()
[Fx] PROVIDE    fx.DotGraph <= go.uber.org/fx.(*App).dotGraph-fm()
[Fx] RUN    supply: stub([]temporal.ServerOption)
[Fx] RUN    provide: go.temporal.io/server/temporal.ServerOptionsProvider()
[Fx] Error returned: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider
    /home/builder/temporal/temporal/fx.go:173:
config validation error: invalid history archival config
[Fx] ERROR        Failed to initialize custom logger: could not build arguments for function "go.uber.org/fx".(*module).constructCustomLogger.func2
    /go/pkg/mod/go.uber.org/fx@v1.20.0/module.go:251:
failed to build fxevent.Logger:
could not build arguments for function "go.temporal.io/server/temporal".glob..func8
    /home/builder/temporal/temporal/fx.go:1026:
failed to build log.Logger:
received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider
    /home/builder/temporal/temporal/fx.go:173:
config validation error: invalid history archival config
Unable to create server. Error: could not build arguments for function "go.uber.org/fx".(*module).constructCustomLogger.func2 (/go/pkg/mod/go.uber.org/fx@v1.20.0/module.go:251): failed to build fxevent.Logger: could not build arguments for function "go.temporal.io/server/temporal".glob..func8 (/home/builder/temporal/temporal/fx.go:1026): failed to build log.Logger: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider (/home/builder/temporal/temporal/fx.go:173): config validation error: invalid history archival config.
Stream closed EOF for temporal/temporal-frontend-57cc658c97-85lnj (temporal-frontend)

This is the config pulled from a config map:

log:
  stdout: true
  level: "debug,info"

persistence:
  defaultStore: default
  visibilityStore: visibility
  numHistoryShards: 512
  datastores:
    default:
      sql:
        pluginName: "postgres"
        driverName: "postgres"
        databaseName: "temporal"
        connectAddr: "temporal-pg-rw.temporal:5432"
        connectProtocol: "tcp"
        user: postgres
        password: "{{ .Env.TEMPORAL_STORE_PASSWORD }}"
        maxConnLifetime: 1h
        maxConns: 20
        secretName: ""
    visibility:
      sql:
        pluginName: "postgres"
        driverName: "postgres"
        databaseName: "temporal_visibility"
        connectAddr: "temporal-pg-rw.temporal:5432"
        connectProtocol: "tcp"
        user: "postgres"
        password: "{{ .Env.TEMPORAL_VISIBILITY_STORE_PASSWORD }}"
        maxConnLifetime: 1h
        maxConns: 20
        secretName: ""

global:
  membership:
    name: temporal
    maxJoinDuration: 30s
    broadcastAddress: {{ default .Env.POD_IP "0.0.0.0" }}

  pprof:
    port: 7936

  metrics:
    tags:
      type: frontend
    prometheus:
      timerType: histogram
      listenAddress: "0.0.0.0:9090"

services:
  frontend:
    rpc:
      grpcPort: 7233
      membershipPort: 6933
      bindOnIP: "0.0.0.0"

  history:
    rpc:
      grpcPort: 7234
      membershipPort: 6934
      bindOnIP: "0.0.0.0"

  matching:
    rpc:
      grpcPort: 7235
      membershipPort: 6935
      bindOnIP: "0.0.0.0"

  worker:
    rpc:
      grpcPort: 7239
      membershipPort: 6939
      bindOnIP: "0.0.0.0"
clusterMetadata:
  enableGlobalDomain: false
  failoverVersionIncrement: 10
  masterClusterName: "active"
  currentClusterName: "active"
  clusterInformation:
    active:
      enabled: true
      initialFailoverVersion: 1
      rpcName: "temporal-frontend"
      rpcAddress: "127.0.0.1:7933"
dcRedirectionPolicy:
  policy: "noop"
  toDC: ""
archival:
  history:
    enableRead: true
    provider:
      s3store:
        region: ca-central-1
    state: enabled
  visibility:
    enableRead: true
    provider:
      s3store:
        region: ca-central-1
    state: enabled

publicClient:
  hostPort: "temporal-frontend:7233"

dynamicConfigClient:
  filepath: "/etc/temporal/dynamic_config/dynamic_config.yaml"
  pollInterval: "10s"

which looks correct. Any ideas why I'm getting this error?

joshbranham commented 11 months ago

At quick glance looks like the namespaceDefaults config where the bucket URI is set (https://github.com/temporalio/temporal/tree/main/common/archiver/s3store) is missing. Probably in the templates for the helm chart

Nepoxx commented 11 months ago

I have tried with and without the namespace defaults but the results doesn't change.

For what it's worth:

apiVersion: v1
data:
  config_template.yaml: |-
   # omitted
    archival:
      history:
        enableRead: true
        provider:
          s3store:
            region: ca-central-1
        state: enabled
      visibility:
        enableRead: true
        provider:
          s3store:
            region: ca-central-1
        state: enabled
    namespaceDefaults:
      archival:
        URI: s3://my-bucket-name-archival
        state: enabled
      history:
        URI: s3://my-bucket-name-archival
        state: enabled
kind: ConfigMap

I still get config validation error: invalid history archival config

(I also tried with a different bucket for archival and history but still get the same error)

Nepoxx commented 11 months ago

My namespaceDefaults were wrong, it should have been:

namespaceDefaults:
  archival:
    visibility:
      URI: s3://my-bucket-name-archival
      state: enabled
    history:
      URI: s3://my-bucket-name-archival
      state: enabled

Sorry about this.

There's a slight typo here: https://github.com/temporalio/temporal/blob/ce211cb2d966b56b4b983043ae2233057f6b3aaf/common/config/archival.go#L57 (should be domain)