hhk7734
commented
3 years ago I don't know anything about metrics.
Closed guysoft closed 3 years ago
hhk7734
commented
3 years ago I don't know anything about metrics.
hhk7734
commented
3 years ago I want to help after checking how metric is moved and used inside tensorflow, but It is difficult because I have a lot of work now.
guysoft
commented
3 years ago Here is a tl;dr
Metrics are like a loss function. They let you calculate about y_true y_pred and get a number. Telling you how well you are doing.
Unlike a loss, a metric does not have to be differentiable. The derivative is needed to calculate gradient descent, but you only need one of those.
So why add a metric? It lets you see how well your network is learning. You might find that your networks epoch 100 is great but 200 its not as good. It also lets you score how well one network is doing compared to another in a real scenario (how many successful detection, for example, not some strange loss definition of success).
You could set your loss as a metric too.
The loss in this repo is set up as a class, that generates function. Its defined here: https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/tf/train.py#L32
Keras has several popular metrics. Here are a list: https://keras.io/api/metrics/
The thing is. I think that because the tensor shape here is (None, None, None, None) its causing issues with metrics.
guysoft
commented
3 years ago Ok, not sure why, but it seems that adding this makes my metrics work:
tf.config.experimental_run_functions_eagerly(True)Update:
This is the non-deprecated way:
tf.config.run_functions_eagerly(True) hi~
TF2 training default mode is Autograph, the others are Eager. On Autograph mode, it seems to be normal that print(shape) is (None, None, None, None).
When calling compile, TF2 make compiled_loss. and automatically registor loss to metrics.
class LossesContainer(Container):
"""A container class for losses passed to `Model.compile`."""
@property
def metrics(self):
"""Per-output loss metrics."""
if not self._built:
return []
per_output_metrics = [
metric_obj for metric_obj in nest.flatten(self._per_output_metrics)
if metric_obj is not None
]
return [self._loss_metric] + per_output_metricsand according to TF docs.
def compile(self,
optimizer='rmsprop',
loss=None,
metrics=None,
loss_weights=None,
weighted_metrics=None,
run_eagerly=None,
steps_per_execution=None,
**kwargs):
"""Configures the model for training.
Arguments:
...
loss: String (name of objective function), objective function or
`tf.keras.losses.Loss` instance.
...
metrics: List of metrics to be evaluated by the model during training
and testing. Each of this can be a string (name of a built-in
function), function or a `tf.keras.metrics.Metric` instance. See
`tf.keras.metrics`.
"""The loss type and the metric type seem to be different.
I made a callback and refactored the loss function to evaluate the training.
https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/tf/training/callbacks/yolo_each_step.py https://github.com/hhk7734/tensorflow-yolov4/blob/master/py_src/yolov4/tf/training/yolo_loss.py
So, training can be evaluated in real time. Ex) https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-training#tensorboard
Hey, I have the following code for training:
However if I add a matric, for example:
or:
And then add it to the model compile:
I get the following error.
How come that setting the loss function as a metric makes this fail? Since AFAIK loss functions should function as metrics too.
Using latest release.
If I make just a blank loss function, I can get it to print the shape of the tensor which is
(None, None, None, None)Thanks,