search

tensorlab / tensorfx

TensorFlow framework for training and serving machine learning models
Apache License 2.0
196 stars 41 forks source link

output vs job-dir #8

Closed brandondutra closed 7 years ago

brandondutra commented 7 years ago

tensorfx.tools.tfx train takes output parameter, but this is mapped to 'job-dir'. What do we gain by hiding the job-dir idea?

nikhilk commented 7 years ago

It says this is where the outputs of a job go more than just job-dir. I'd like to see this eventually:

%tfx train
input:
  train: ...
  eval: ...
output: ...

So input/output of a job, which is user visible whereas job-dir is fully internally handled in the service/framework, hence the choice to deviate (while on the service its consistent with job* ... all of which is not user facing unless you use the MLEngine API directly).

Note: I think these input arguments should be surfaced via the job object as a result of argparsing (rather than on args which is meant to represent hyperparams) to land this notion of input/output of a job.

nikhilk commented 7 years ago

Changed some APIs and implementation around inputs/outputs to have them appear side-by-side and associated argparsing.

Also the argparsing accepts both --output and --job-dir to allow for using --output in both direct launch scenario and tfx train launch scenario (and noted --job-dir is for ML Engine compat only).