nasa / delta

Deep Learning for Satellite Imagery
Apache License 2.0
210 stars 65 forks source link

DELTA throws error when classifying images without labels #126

Closed mvonpohle closed 3 years ago

mvonpohle commented 3 years ago

I think in the ROI metric changes there's a bug where if you try to classify some images without any associated labels (like I'm currently doing for the cloud shadow) you get an error in classify.py.

Here's the specific Traceback:

Traceback (most recent call last):
  File "/home/mvonpohl/miniconda3/envs/delta/bin/delta", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/usr/local/home/mvonpohl/projects/delta_project/delta/bin/delta", line 25, in <module>
    sys.exit(main.main(sys.argv))
  File "/usr/local/home/mvonpohl/projects/delta_project/delta/delta/subcommands/main.py", line 50, in main
    return options.function(options)
  File "/usr/local/home/mvonpohl/projects/delta_project/delta/delta/subcommands/commands.py", line 28, in main_classify
    classify.main(options)
  File "/usr/local/home/mvonpohl/projects/delta_project/delta/delta/subcommands/classify.py", line 368, in main
    cm, metrics, full_metrics = classify_image(model, this_image,
TypeError: cannot unpack non-iterable NoneType object

Line 368 is expecting a Tuple return:

cm, metrics, full_metrics = classify_image(model, this_image,
                                                       labels.load(i) if labels else None,
                                                       image_path, net_name, options,
                                                       shapes, persistent_metrics=full_metrics)

But classify_image (lines 207-219) can return None if there are no labels:


    if label:
        cm = predictor.confusion_matrix()
        class_names = list(map(lambda x: x.name, config.dataset.classes))
        if len(config.dataset.classes) != cm.shape[0]:
            class_names = list(map(lambda x: 'Class %d' % (x), range(cm.shape[0])))
        if options.confusion and (not shapes):
            save_confusion(cm, class_names,
                           os.path.join(out_path, 'confusion_' + os.path.splitext(base_out)[0] + '.pdf'))
        metrics = predictor.metrics()
        metrics, persistent_metrics = (metrics[0:num_temp_metrics], metrics[num_temp_metrics:])
        return cm, metrics, persistent_metrics
    return None

You can replicate this by running a trained model using this config (wv_validate.yaml):

dataset:
  images:
    type: tiff
    nodata_value: 0.0
    directory: /usr/local/irg/data/wv/validate

io:
  tile_size: [2048, 2048]

and this command:

delta classify --config wv_validate.yaml --prob --overlap 32 E102_WV_S1.1_nopool.h5
mvonpohle commented 3 years ago

Just checked and fetched the repository right before PR #96 and seems to work with the same command I mentioned above.

ScottMcMichael commented 3 years ago

This should be fixed in the latest PR I merged.

mvonpohle commented 3 years ago

Yep! It's working for me too! Thanks @ScottMcMichael!