metagpt.config.NotConfiguredException: Set OPENAI_API_KEY first

[gnoejuan]

Greetings!

I've tried several ways of passing the OPENAI_API_KEY

First, here is the Dockerfile:

Dockerfile

```Dockerfile FROM bitnami/git:latest as clone RUN git clone https://github.com/geekan/MetaGPT.git FROM bitnami/node:latest COPY --from=clone MetaGPT/ . RUN python3 -m pip install --upgrade pip setuptools wheel RUN npm install -g @mermaid-js/mermaid-cli &&\ npm cache clean --force RUN pip install -r requirements.txt &&\ pip cache purge &&\ python3 setup.py install ```

I tried with the equivalent of the docker run command in the instructions.

nerdctl run --rm -v /home/charlesrod/Documents/Projects/MetaGPT/config:/app/metagpt/config:rw -v /home/charlesrod/Documents/Projects/MetaGPT/workspace:/app/metagpt/workspace:rw docker.io/library/metagpt:latest

Oh, here is the config.yaml / key.yaml with the api key redacted

key.yaml

```yaml # DO NOT MODIFY THIS FILE, create a new key.yaml, define OPENAI_API_KEY. # The configuration of key.yaml has a higher priority and will not enter git #### if OpenAI OPENAI_API_KEY: "sk-----" #OPENAI_API_BASE: "YOUR_API_BASE" OPENAI_API_MODEL: "gpt-4" MAX_TOKENS: 1500 RPM: 10 #### if AZURE, check https://github.com/openai/openai-cookbook/blob/main/examples/azure/chat.ipynb #OPENAI_API_TYPE: "azure" #OPENAI_API_BASE: "YOUR_AZURE_ENDPOINT" #OPENAI_API_KEY: "YOUR_AZURE_API_KEY" #OPENAI_API_VERSION: "YOUR_AZURE_API_VERSION" #DEPLOYMENT_ID: "YOUR_DEPLOYMENT_ID" #### for Search ## Visit https://serpapi.com/ to get key. #SERPAPI_API_KEY: "YOUR_API_KEY" ## Visit https://console.cloud.google.com/apis/credentials to get key. #GOOGLE_API_KEY: "YOUR_API_KEY" ## Visit https://programmablesearchengine.google.com/controlpanel/create to get id. #GOOGLE_CSE_ID: "YOUR_CSE_ID" #### for TTS #AZURE_TTS_SUBSCRIPTION_KEY: "YOUR_API_KEY" #AZURE_TTS_REGION: "eastus" ```

And when I check for the file, its there.

nerdctl run --rm -v /home/charlesrod/Documents/Projects/MetaGPT/config:/app/metagpt/config:rw -v /home/charlesrod/Documents/Projects/MetaGPT/workspace:/app/metagpt/workspace:rw docker.io/library/metagpt:latest cat metagpt/config/key.yaml

# DO NOT MODIFY THIS FILE, create a new key.yaml, define OPENAI_API_KEY.
# The configuration of key.yaml has a higher priority and will not enter git

#### if OpenAI

OPENAI_API_KEY: "sk-

Running without the volume mounts returns this:

nerdctl run --rm docker.io/library/metagpt:latest cat ./config/config.yaml

# DO NOT MODIFY THIS FILE, create a new key.yaml, define OPENAI_API_KEY.
# The configuration of key.yaml has a higher priority and will not enter git

#### if OpenAI

OPENAI_API_KEY: "YOUR_API_KEY"
#OPENAI_API_BASE: "YOUR_API_BASE"
OPENAI_API_MODEL: "gpt-4"
MAX_TOKENS: 1500
RPM: 10

but that's cause I cloned from source.

I also tried with passing the api key as an environment argument to the run command.

nerdctl run --rm -e OPENAI_API_KEY="sk-----" -v /home/charlesrod/Documents/Projects/MetaGPT/workspace:/app/metagpt/workspace docker.io/library/metagpt:latest python startup.py

I created a Kubernetes resource to run it. One where the config is mounted and another where the environment variables are part of the job spec ( latest attempt before raising this issue )

Job.yaml

```yaml apiVersion: v1 kind: ConfigMap metadata: name: meta-gpt-config data: TASK_ARGUMENT: "Write a cli snake game" --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: metagpt-pv-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi --- apiVersion: batch/v1 kind: Job metadata: name: metagpt-job spec: template: spec: restartPolicy: Never volumes: - name: metagpt-pv-storage persistentVolumeClaim: claimName: metagpt-pv-claim containers: - name: gpt-agency image: metagpt:latest imagePullPolicy: Never volumeMounts: - mountPath: "/app/metagpt/workspace" name: metagpt-pv-storage command: ["/opt/bitnami/python/bin/python", "startup.py"] args: [ "$(TASK_ARGUMENT)" ] env: - name: TASK_ARGUMENT valueFrom: configMapKeyRef: name: meta-gpt-config key: TASK_ARGUMENT - name: OPENAI_API_KEY value: "sk-------" - name: OPENAI_API_MODEL value: "gpt-4" - name: MAX_TOKENS value: "1500" - name: RPM value: "10" - name: sidecar-container image: busybox volumeMounts: - mountPath: "/app/metagpt/workspace" name: metagpt-pv-storage command: ["sh", "-c", "while true; do sleep 3600; done"] ```

And the pod Errors with "metagpt.config.NotConfiguredException: Set OPENAI_API_KEY first".

Image


Updating the job to command: ["printenv"] again proves that teh env is there

2023-07-11T04:53:01.613165273Z HOSTNAME=metagpt-job-5z8hl
2023-07-11T04:53:01.613166605Z OS_ARCH=amd64
2023-07-11T04:53:01.613167610Z OS_FLAVOUR=debian-11
2023-07-11T04:53:01.613168404Z OS_NAME=linux
2023-07-11T04:53:01.613169287Z APP_VERSION=20.4.0
2023-07-11T04:53:01.613170462Z BITNAMI_APP_NAME=node
2023-07-11T04:53:01.613171725Z MAX_TOKENS=1500
2023-07-11T04:53:01.613172920Z RPM=10
2023-07-11T04:53:01.613174496Z TASK_ARGUMENT=Write a cli snake game
2023-07-11T04:53:01.613177938Z OPENAI_API_KEY=sk---------
2023-07-11T04:53:01.613179042Z OPENAI_API_MODEL=gpt-4

From the code in config.py https://github.com/geekan/MetaGPT/blob/317f0955ab81919e2ff32f0a81ad112131965123/metagpt/config.py#L64C73-L64C73

It looks like it should grab the environment variable as well. I rebuilt the image, since the git clone stage was cached from 2 days ago at this point.

I'm stumped at the moment.

[geekan]

@voidking

[voidking]

@gnoejuan I'm sorry I couldn't locate the problem according to your description. I have a few suggestions that might help:

1. Make sure your workdir in container is /app/metagpt which contains the MetaGPT codes, and exec python startup.py "something" in the /app/metagpt directory.
2. Make sure the config directory is in workdir. The correct path like this: /app/metagpt/config/key.yaml. Then the program can find the config.
3. Change the config content by mounting a file or sed in Dockerfile. Make sure the program can work by configfile.
4. Give parameters through environment variables.

By the way, you can debug your container by the following step:

docker run --name test -d docker.io/library/metagpt:latest tail -f /dev/null
docker exec -it test /bin/sh
cat config/key.yaml
python startup.py "do something"
export OPENAI_API_KEY="xxx"
python startup.py "do something"
...

Then you can find where the problem is.

[gnoejuan]

Awesome! Thank you!

So, looks like with the command line I was running earlier, the mount was in the wrong place! Good catch!

But, before I figured out the mount issue, when I was testing the

export OPENAI_API_KEY="xxx"
python startup.py "do something"

step, I ran into the NotConfiguredException.

So, I'm happy and taken care of ( I can modify the Job to use the config ), the documentation in the Readme does suggest that an environment variable should also be a viable approach.

[geekan]

I know the problem, because config.yaml has higher priority, so the stub in config.yaml blocks the env variable. I will fix this in the next update

[geekan]

https://github.com/geekan/MetaGPT/commit/da11d383e60bb4d1562ed08b9db2ff9ea5721328 pushed. check it again.

[gnoejuan]

Yup! That worked! The Job was able to complete using the environment variables!