ResourceManager should save evaluation results

[galshachaf]

evaluation takes a lot of time, should be able to load it from previous runs. when we run calc_full_rebustness_results, we get a "results" dict. We should add an "evaluation_results" attribute to ResourceManager per model, which will initialize to an empty dict. When trying to get results per layer_indices_list, we first try to look it in this dictionary. If not found, we will evaluate and update this dict. (and save it)

[dorimedini]

@galshachaf Could you give me some example use case? Give me some python pseudocode so I know how you want to use this with existing classes. Note that the way I see it there are 3 (mutually exclusive) options:

• Run clean experiment from scratch (train + evaluate)
• Re-evaluate existing trained models
• Re-generate heatmaps for existing data

How do you want to control when the experiment loads previous data or not?

[dorimedini]

Output a single dict JSON file, load it once and update throughout the experiment