bdzyubak / tensorflow-sandbox

A repository for studying applications of Deep Learning across fields, and demonstrating samples of my code and project managment
0 stars 0 forks source link

Make module to compare training results of different networks #18

Closed bdzyubak closed 2 years ago

bdzyubak commented 2 years ago

Build a module that will take learning histories of several trained network and plot a comparison in terms of: 1) Final Loss. 2) Another argument-specified final metric such as 'dice' or 'accuracy.'
3) Model size. 4) Model training time.

The output should probably be a series of bar charts. Consider sorting by quality (accuracy?) and, if to many networks are specified to plot on one chart, splitting into multiple charts. The module should be callable with a top directory and a list of networks to compare, and should fetch results from subfolders named with the network names.

bdzyubak commented 2 years ago

A script-style module compare_builtin_modules.py was introduced in 14334057314d833accabbf2034015878b15f1299. Currently supports reading a list of metrics from a history csv file based on column names.

Should be made into a class utility to apply to run comparison on any models.

Need to implement custom metric which records model training time.

bdzyubak commented 2 years ago

Added in 5b79cfde9191c33a8bab6a0bb6589713306f6b1b

bdzyubak commented 2 years ago

Sample output for positive metric (dice): positive_dice_metric

Sample output for negative metric (loss): negative_loss_metric

Sample output for training time: training_time_metric