LLNL / merlin

Machine Learning for HPC Workflows
MIT License
119 stars 26 forks source link

Unable to Place Environment Varibles in field resources.workers.sim_workers.nodes of merlin section #433

Closed lvandeca closed 2 months ago

lvandeca commented 1 year ago

Bug Report

Describe the bug In the field resources.workers.<worker_type>.nodes placing environment variables causes an error that the variable is not of type 'null' or does not conform to the format '$(VARNAME)' even when the variable is in the correct format.

To Reproduce To reproduce the behavior, place any environment variable in the node field of the resources section as shown below:

env:
    variables:
        NODES: 1
merlin:
    resources:
        task_server: celery
        workers:
            simworkers:
                steps: [simulations]
                args: -l ERROR --concurrency 32 --prefetch-multiplier 1 -O fair
                nodes: $(NODES)
                machines: [quartz]

Expected behavior Merlin should build the DAG, generate any samples, and the value assigned to $(NODES) should be substituted in.

Screenshots The error that is given when executing merlin run <file>.yaml

image

bgunnar5 commented 2 months ago

This issue is resolved in #484