aws / copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
https://aws.github.io/copilot-cli/
Apache License 2.0
3.53k stars 417 forks source link

[Bug]: Error unmarshalling when variable contains a string of a string array #5907

Open CharlieDigital opened 3 months ago

CharlieDigital commented 3 months ago

Description:

Receiving the following error when using an environment variable which is a string of a string array (JSON array string)

Possibly related:

unmarshal manifest for Load Balanced Web Service: unmarshal "variables": cannot unmarshal field to a string or into a map

Details:

Per Neo4j documentation, the environment variable NEO4J_PLUGINS accepts a value which is a string like so:

docker run -it --rm \
  --publish=7474:7474 --publish=7687:7687 \
  --env NEO4J_AUTH=none \
  --env NEO4J_PLUGINS='["apoc", "graph-data-science"]' \
  neo4j:5.22.0

See: https://neo4j.com/docs/operations-manual/current/docker/plugins/

When specifying this as a variable in manifest.yaml:

variables:
  NEO4J_apoc_export_file_enabled: true
  NEO4J_apoc_import_file_enabled: true
  NEO4J_apoc_import_file_use__neo4j__config: true
  NEO4J_PLUGINS: '["apoc", "apoc-extended", "graph-data-science"]'
  NEO4J_dbms_security_procedures_unrestricted: apoc.*,gds.*,algo.*,spatial.*
  NEO4J_server_directories_neo4j__home: /var/efs/neo4j 

Yields the error as described.

Observed result:

➜  neo4j-poc git:(main) ✗ copilot svc deploy \
  --app neo4j-poc \
  --name neo4j-db \
  --env beta
✘ unmarshal service neo4j-db manifest: unmarshal manifest for Load Balanced Web Service: unmarshal "variables": cannot unmarshal field to a string or into a map

Expected result:

The environment variable is set as expected.

Debugging:

Commenting out this line allows the container to be deployed, but with this line uncommented, the deploy fails during unmarshalling of the config.

Lou1415926 commented 3 months ago

Hi! Can you try NEO4J_PLUGINS: "['apoc', 'apoc-extended', 'graph-data-science']" and let me know if it helps?

CharlieDigital commented 3 months ago

@Lou1415926 , unfortunately this does not work because the underlying code is expecting valid JSON.

I will try a few other variations and see:

# Doesn't work; fails with validation error
NEO4J_PLUGINS: "[\"apoc\", \"apoc-extended\", \"graph-data-science\"]"

# Isn't recognized by the application; ignored
NEO4J_PLUGINS: "['apoc', 'apoc-extended', 'graph-data-science']"

If I manually correct it in the console by editing the JSON:

{
    "name": "NEO4J_PLUGINS",
    "value": "[\"apoc\", \"apoc-extended\", \"graph-data-science\"]"
},

This is OK

Other combinations also yield the incorrect JSON:

NEO4J_PLUGINS: "'["apoc", "apoc-extended", "graph-data-science"]'"

NEO4J_PLUGINS: "'[\"apoc\", \"apoc-extended\", \"graph-data-science\"]'"

The other alternative is to put it into an SSM parameter and map it to the container as a secret

secrets:
  NEO4J_PLUGINS: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/NEO4J_PLUGINS

Where the value of the secret would be a typical JSON array:

["apoc", "apoc-extended", "graph-data-science"]
Lou1415926 commented 3 months ago

ah, I think I know what's wrong now. Can you try

NEO4J_PLUGINS: |
    ["apoc", "apoc-extended", "graph-data-science"]

This unmarshals this chunk of manifest into yaml literal style so that your string is passed on literally.