issue with using external_source_url

[theeheng]

hi there, we are trying to use terraform to deploy the opal server with the latest version on the helm chart and notice that you have committed this changes https://github.com/permitio/opal-helm-chart/commit/25731f62c245e5c55a0faf4f6a8f5178438c33dc recently to change the the default values file 'dataConfigSources' section:

server:
  .........
  dataConfigSources:
    # Option #1 - No data sources
    config:
      entries: []

which currently cause an issue when we set the value for external_source_url as follow:

server:
  dataConfigSources:
     external_source_url: "http://xxx-xxx-xx-xxx"

When we deploy via terraform it will set the environment variable OPAL_DATA_CONFIG_SOURCES to {"config":{"entries":[]},"external_source_url":"http://xxx-xxx-xx-xxx"} and opal server is throwing the following error:

pydantic.error_wrappers.ValidationError: 1 validation error for ServerDataSourceConfig root you must provide ONLY ONE of these fields: config, external_source_url (type=value_error)

I think thats why the default values file was initially set with external_source_url as the first options rather than having the config -> entries as the first options.

I'm wondering if you able to revert this that part of the code changes in the value file or update the server template file to fix this issue.

[RazcoDev]

Hey @theeheng, That's very inconvenient. I've just released a quick hotfix for this, let me know if this handles your issue. Here's the release - https://github.com/permitio/opal-helm-chart/releases/tag/0.0.21

By the way, you can also join our Slack community so we can communicate their !

[theeheng]

I just did a quick test using the latest hotfix 0.0.21, it still set OPAL Server the environment variable OPAL_DATA_CONFIG_SOURCES as

OPAL_DATA_CONFIG_SOURCES: {"config":{"entries":[]},"external_source_url":"http://www.external_source_url.com"}

where the correct value for the environment variable should be the following when using external source url only:

OPAL_DATA_CONFIG_SOURCES: {"external_source_url":"http://www.external_source_url.com"}

here is the screenshot when i describe the pod after doing helm install

and when i check the log for the opal server pod to see why its crashing it shows "you must provide ONLY ONE of these fields: config, external_source_url (type=value_error)"

[RazcoDev]

Hey @theeheng , That's odd, I tried the helm on my local and it fixed this issue. I'll look at it again and fix this ASAP.

[RazcoDev]

For me it looks good, maybe i'm missing something ? I'll share with you my configuration - my Values.yaml file -

image:
  client:
    registry: docker.io
    repository: permitio/opal-client
  server:
    registry: docker.io
    repository: permitio/opal-server
  pgsql:
    registry: docker.io
    repository: postgres
    tag: alpine

server:
  port: 7002
  policyRepoUrl: "https://github.com/permitio/opal-example-policy-repo"
  policyRepoSshKey: null
  policyRepoClonePath: null
  policyRepoMainBranch: null
  pollingInterval: 30
  dataConfigSources: 
    external_source_url: "http://www.external_source_url.com"

And here is the output when I run helm template ./ | grep DATA_CONFIG -A 1 :

            - name: OPAL_DATA_CONFIG_SOURCES
              value: '{"external_source_url":"http://www.external_source_url.com"}'

[theeheng]

I just tried with the sample value file you provided and name it as values.Permitio.yaml with the following content:

image:
  client:
    registry: docker.io
    repository: permitio/opal-client
  server:
    registry: docker.io
    repository: permitio/opal-server
  pgsql:
    registry: docker.io
    repository: postgres
    tag: alpine

server:
  enabled: true
  port: 7002
  policyRepoUrl: "https://github.com/permitio/opal-example-policy-repo"
  policyRepoSshKey: null
  policyRepoClonePath: null
  policyRepoMainBranch: null
  pollingInterval: 30
  dataConfigSources: 
    external_source_url: "http://www.external_source_url.com"

client: 
  enabled: false

I had minukube running on my machine and I run helm install with your latest changes in the opal-helm-chart repo :

helm install auth opal-helm-chart/ --values opal-helm-chart/values.Permitio.yaml

when I check my auth server pod using kubectl describe pods/xxxxxxxx, it still says :

OPAL_DATA_CONFIG_SOURCES: {"config":{"entries":[]},"external_source_url":"http://www.external_source_url.com"}

I think you need to do helm install to actually see that the environment variable actually set incorrectly:

[RazcoDev]

I understand what happens here. Because the default values.yaml includes the config.entries it merges with your value.Permit.yaml file. To override the default values.yaml, set this -

  dataConfigSources:
    config: null
    external_source_url: "https://your-api.com/path/to/api/endpoint"

Hope it helps.

[theeheng]

I get the following error when i set the config to null :

Error: values don't meet the specifications of the schema(s) in the following chart(s):
opal:
- server.dataConfigSources.config: Invalid type. Expected: object, given: null

I think its because in the schema json file it set the config type as object :

[RazcoDev]

That's really odd. What do you think about setting us a quick session to solve this thing ? I'm sure that it'll be much more efficient :)

Let me know if that works for you - you can also just grab a spot on my Calendly - https://calendly.com/razco/1-on-1-permit-io