search

tensorflow / adanet

Fast and flexible AutoML with learning guarantees.
https://adanet.readthedocs.io
Apache License 2.0
3.47k stars 529 forks source link

Accessing the Adanet loss for each candidate subnetwork? #120

Open nicholasbreckwoldt opened 4 years ago

nicholasbreckwoldt commented 4 years ago

Hi there,

I would like to be able to access and record the individual Adanet losses returned by the adanet.Evaluator during evaluation of each subnetwork (i.e. candidate) within a given iteration.

Is this currently possible, and if so, how would one go about accessing these losses?

EugenHotaj commented 4 years ago

Hi @nicholasbreckwoldt,

As far as I know, there is no out of the box way to get the subnetwork losses from the AdaNet estimator. For example, when calling estimator.evaluate(), only the metrics for the final ensemble are returned.

However, if you're only interested in tracking the losses during training, we do write them out as summaries for each subnetwork, which can be viewed via Tensorboard. If, instead, you're interested in computing something based on these losses, this would require modifying the AdaNet code to use the losses however you need to.