For heatmap, save efficient numpy files for only the classes in config

[arpgh]

Two issues: (for _save_videoheatmap: true in config)

• .npy files are being saved for all classes in numpy folder even though only a few classes are specified in config. This is very inefficient for runtime and storage.
• Each .npy file saved is 5.5MB in size which indicates more data is being saved from frame than necessary. Is whole array being saved?
• (numpy folder may only be made if _save_videoheatmap is true. Currently, it is always being made.)

[regmiz]

Fixed in 146f44f476653eafbbaee38c3fbde3ef0e09e87c at branch Issue85.

[arpgh]

Is snappy compressed json being read properly?

Branch: issue85 Path: gpu2:/mnt/data/wc14cls/training/0seed/vdos/wc14-ChiAus-1stHalf Video: wc14-ChiAus-1stHalf.mkv Json used: json-curation Log with error: runlog-process_video-heatmap.txt-ERR Key configs: post process, save heatmap for 2 classes

[regmiz]

This seems ok for me. Can you try again?

one thing that I noticed was that in the config.yaml in the folder gpu2:/mnt/data/wc14cls/training/0seed/vdos/wc14-ChiAus-1stHalf, the value

save_video_heatmap: heatmap

doesn't look correct.

The fix to read .snappy files is already present is the branch.

[regmiz]

Also, please update to the latest issue85 branch

[arpgh]

When running caffe & post process in parallel with save heatmap, the log says - Frame 8001, heatmap/localization computed for all nonBackgroundClassIds. Is this correct?

Localization is calculated only for those classes with patch score above threshold (as in closed issue#78), and heatmap only for those classes specified in config (11 & 27 in this case) ... right? Looking at the time taken for this processing, it looks all non background classes are being calculated.

Path: gpu2:/mnt/data/wc14cls/training/0seed/vdos/test-issue85

[arpgh]

Heatmap videos generated are incorrect. Here are some example frames for class 27 (hyundai) - Path: gpu2:/mnt/data/wc14cls/training/0seed/vdos/test-issue85/heatmapVdos/

[regmiz]

One more version of xStride --> yStride bug Fixed at d941745615ed32b3b8419631467a88c34a2b5639 in issue85 branch.

[regmiz]

At 474950ab0f1990f74ac2f761be33f79e7bc216dd in issue85 branch - the heatmaps saving as well as localization computation should be limited to only those classes listed in the config.yaml plus classes above threshold.

If curation is enabled, a few more classes will be added to post-processing.

Further optimization of this logic is tracked in https://github.com/zigvu/khajuri/issues/89. The above changes gets us most of the benefits we need when creating heatmap videos.

[regmiz]

Eg.

{FramePostProcessor.py::47::2015-02-09--01:51:32} DEBUG PID:27896 - Frame 41, heatmaps saved for classes [1, 6], curations computed for classes [10, 13, 15, 17, 23, 44, 30, 43, 40, 0, 2, 4], localizations computed for classes [13, 15, 44, 30, 43, 4]

This tells us that the list of classes for post-processing will be:

set( [ 1, 6 ] + [10, 13, 15, 17, 23, 44, 30, 43, 40, 0, 2, 4] + [13, 15, 44, 30, 43, 4 ] )

[arpgh]

Calculating new cellMap and saving to file /tmp/savedBoundaries.p

Noticed this even though no change in scale or stride in config. Does this need update from regular post processing to use the existing file?

[arpgh]

Heatmaps now look correct. Here are the same two frames from above -

Also, take a look at frame below which shows effect of finer scale & strides on the localization ... (we'll need to evaluate this inter-play to achieve good localization in a separate thread)

[arpgh]

This is working now when verified with multiple classes. Can be merged.

[regmiz]

Merged into development at 474950ab0f1990f74ac2f761be33f79e7bc216dd.

[arpgh]

Reopening this due to classes seen in localization array for many frames in log file. Here's an example -

Frame 13436, heatmaps saved for classes [22], curations computed for classes [], localizations computed for classes [24, 27, 15, 16, 44, 43, 40, 4]

4, 15, 43, 44 are background classes, so shouldn't be used in localization. Are these classes actually being used to calculate localization or just in log?

Path: gpu2:/mnt/data/wc14cls/training/0seed/vdos/test-issue85-issue79/

[regmiz]

The issue is only with the log message

[regmiz]

so, localization is not calculated for background classes

[regmiz]

Fixed at b91da24749b4f4031ed1e18ae9c7153306a30984 in branch issue85_reop

[arpgh]

Tested ... checked log and compared json produced. Looks good.