search

microsoft / ADBench

Benchmarking various AD tools.
MIT License
107 stars 40 forks source link

Tensorflow Hand module #139

Closed mikhailnikolaev closed 5 years ago

mikhailnikolaev commented 5 years ago

Tensorflow Hand objective differentiation test class is implemented.

Note: Tensorflow 2.0 is used. All calculations are made in eager execution mode.

Graphs

Jacobian

HAND (Complicated, Small) Jacobian - Release Graph HAND (Simple, Big) Jacobian - Release Graph HAND (Simple, Small) Jacobian - Release Graph HAND (Complicated, Big) Jacobian - Release Graph

Jacobian ÷ objective

HAND (Simple, Big) Jacobian ÷ objective - Release Graph HAND (Simple, Small) Jacobian ÷ objective - Release Graph HAND (Complicated, Big) Jacobian ÷ objective - Release Graph HAND (Complicated, Small) Jacobian ÷ objective - Release Graph

Objective

HAND (Complicated, Small) Objective - Release Graph HAND (Simple, Big) Objective - Release Graph HAND (Simple, Small) Objective - Release Graph HAND (Complicated, Big) Objective - Release Graph

Objective ÷ manual

HAND (Simple, Big) Objective ÷ manual - Release Graph HAND (Simple, Small) Objective ÷ manual - Release Graph HAND (Complicated, Big) Objective ÷ manual - Release Graph HAND (Complicated, Small) Objective ÷ manual - Release Graph
mikhailnikolaev commented 5 years ago

@awf, @tomjaguarpaw, I've moved us jacobian part merging into two columns to output function of the test class. New plots are attached to the PR. I attach old plots to this message for comparision. Note, that I've merged PyTorch new Hand version to the branch, so, little changes in PyTorch plot behaviour are correct. Also, new X-axis for Hand is used. As you can see there are not any significant changes in time for Tensorflow after refactoring. But I like the new version of the code, because it becames clearer, so, I've added the commit with changes to the PR.

Plots

Note: there are only graphs for complicated jacobian calculating, because only these plots could be changed.

Jacobian

HAND (Complicated, Big) Jacobian - Release Graph HAND (Complicated, Small) Jacobian - Release Graph

Jacobian ÷ objective

HAND (Complicated, Big) Jacobian ÷ objective - Release Graph HAND (Complicated, Small) Jacobian ÷ objective - Release Graph
mikhailnikolaev commented 5 years ago

This looks excellent to me. Did we discuss it last Wednesday?

Yes we did. Also we wanted to change code a bit and create new plots. That has been done

tomjaguarpaw commented 5 years ago

Good, well I'm happy with it. I think we should merge it. If @awf has comments then we can address them post-merge.