wandb / client-ng

Experimental wandb CLI and Python API - See Experimental section below.
http://wandb.com
16 stars 7 forks source link

add args_no_hyphens, args_json, args_json_file to sweep command #224

Closed raubitsj closed 3 years ago

raubitsj commented 3 years ago

WB-3359 #close add args_no_hyphens to sweep command

User request: https://wandb.atlassian.net/browse/WB-3359

raubitsj commented 3 years ago

Sure. That was the plan eventually.

I wasn't sure how people would want it tho.

I'll add args_json and args_json_file Where the latter writes a file and gives you back the filename

raubitsj commented 3 years ago

May as well add args_json while we're at it 😎

Added, please take another look

This sweep config is fun:

program: train-no-hyphens.py
method: grid
# Parameters to search over
parameters:
  optimizer:
    values: ["sgd", "adam"]
  flag1:
    values:
    - false
    - true
  flag2:
    values:
    - false
    - true
command:
    - ${env}
    - python
    - ${program}
    - ${args_no_hyphens}
    - ${args}
    - ${args_json}
    - ${args_json_file}

The filename is relative right now. which i think is fine, but could cause some issues, I want to leave it that way until looking into file path issues (there is a ticket for it already)

This is the result:

ARG 0 train-no-hyphens.py
ARG 1 flag1=True
ARG 2 flag2=True
ARG 3 optimizer=adam
ARG 4 --flag1=True
ARG 5 --flag2=True
ARG 6 --optimizer=adam
ARG 7 {"flag1": true, "flag2": true, "optimizer": "adam"}
ARG 8 wandb/sweep-9l51kef7/config-7duf503q.json
coveralls commented 3 years ago

Pull Request Test Coverage Report for Build 2680a018-b8b2-478d-84fc-4769726a73cb


Changes Missing Coverage Covered Lines Changed/Added Lines %
wandb/wandb_agent.py 0 11 0.0%
<!-- Total: 0 11 0.0% -->
Files with Coverage Reduction New Missed Lines %
tests/utils/mock_server.py 1 84.65%
wandb/apis/internal_runqueue.py 1 25.45%
wandb/internal/internal.py 1 27.89%
wandb/internal/meta.py 1 72.39%
wandb/internal/run.py 1 61.54%
wandb/lib/ipython.py 1 71.43%
wandb/old/core.py 1 57.63%
wandb/sdk/wandb_config.py 1 69.72%
wandb/sdk/wandb_login.py 1 91.04%
wandb/sweeps/base.py 1 80.0%
<!-- Total: 969 -->
Totals Coverage Status
Change from base Build 8ac2c934-d164-4f26-b5a6-7beeae39c9e6: -4.1%
Covered Lines: 12933
Relevant Lines: 21196

💛 - Coveralls