search

microsoft / mwt-ds

Umbrella repository for projects related to the MWT Decision Service
187 stars 77 forks source link

Output policy arguments from hyper parameter sweep should not contain --dsjson in it #148

Closed sharathmalladi closed 4 years ago

sharathmalladi commented 5 years ago

Since we don't expect custom policies to have dsjson in the policy arguments, we don't want to output policies that have the dsjson in the policy arguments as well.

marco-rossi29 commented 5 years ago

Can you specify what your definition of Output policy arguments from hyper parameter sweep is? In particular, can you post a link to a line?

sharathmalladi commented 5 years ago

I was refering to this line of code: https://github.com/microsoft/mwt-ds/blob/master/DataScience/Experimentation.py#L210

When the offline experimentation results are saved, the output policies also list "--dsjson" in them. We don't want this argument being output since it is not really a ML argument.

marco-rossi29 commented 5 years ago

Then I don't agree with this issue: Experimentation.py is looking for the best VW command and --dsjson must be part of such command, otherwise VW wouldn't work