Improve DiffRules integration and tests

[devmotion]

This PR fixes some problems with the DiffRules integration and its tests. It is needed for https://github.com/JuliaDiff/DiffRules.jl/pull/79 (relevant DiffRules tests pass with that PR).

Mainly, the PR

• disables DiffRules definitions and tests for functions with complex outputs and derivatives as they are not supported by ReverseDiff
• adds support for NaN comparisons to the tests (necessary since in DiffRules undefined and non-existing derivatives are implemented as NaN and hence otherwise comparisons with ForwardDiff will fail if both return NaN)
• changes the tests such that they compare ReverseDiff results with the results of the corresponding ForwardDiff calls (currently, e.g., derivatives wrt to two arguments in ReverseDiff are compared with two separately computed ForwardDiff derivatives even though internally both derivatives are computed with one ForwardDiff call - this causes test errors since e.g. if one derivative is NaN the ForwardDiff results of both approaches are different for the derivative of the other argument, one will be NaN and one might not)
• improves the map/broadcasting of DiffRules as currently internally derivatives are computed with ForwardDiff always for both arguments, even if only one is tracked (this was uncovered by the changes to the tests mentioned above and it ensures that derivatives of functions where derivatives are defined only for one argument return non-NaN results, as in ForwardDiff)

The vcat test error is unrelated and also present on the master branch and other PRs. Edit: Fixed on the master branch,

I also assume we could do better than ForwardDiff here and also avoid that all results become NaN if derivatives are computed with respect to both arguments and only one is defined/exists. But replacing ForwardDiff with a direct implementation of the DiffRules-derivatives seemed to require much larger changes, and I tried to apply only a somewhat minimal set of changes required for https://github.com/JuliaDiff/DiffRules.jl/pull/79.

[codecov-commenter]

Codecov Report

Base: 85.16% // Head: 81.24% // Decreases project coverage by -3.92% :warning:

Coverage data is based on head (088182c) compared to base (8ac1f7d). Patch coverage: 58.06% of modified lines in pull request are covered.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #209 +/- ## ========================================== - Coverage 85.16% 81.24% -3.93% ========================================== Files 18 18 Lines 1861 1578 -283 ========================================== - Hits 1585 1282 -303 - Misses 276 296 +20 ``` | [Impacted Files](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff) | Coverage Δ | | |---|---|---| | [src/ReverseDiff.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL1JldmVyc2VEaWZmLmps) | `100.00% <ø> (ø)` | | | [src/derivatives/scalars.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2Rlcml2YXRpdmVzL3NjYWxhcnMuamw=) | `95.74% <ø> (-0.98%)` | :arrow_down: | | [src/derivatives/elementwise.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2Rlcml2YXRpdmVzL2VsZW1lbnR3aXNlLmps) | `74.71% <58.06%> (-4.16%)` | :arrow_down: | | [src/derivatives/broadcast.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2Rlcml2YXRpdmVzL2Jyb2FkY2FzdC5qbA==) | `76.62% <0.00%> (-13.99%)` | :arrow_down: | | [src/tape.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL3RhcGUuamw=) | `55.55% <0.00%> (-9.16%)` | :arrow_down: | | [src/tracked.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL3RyYWNrZWQuamw=) | `86.81% <0.00%> (-5.49%)` | :arrow_down: | | [src/api/hessians.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2FwaS9oZXNzaWFucy5qbA==) | `84.00% <0.00%> (-3.50%)` | :arrow_down: | | [src/api/tape.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2FwaS90YXBlLmps) | `72.22% <0.00%> (-3.05%)` | :arrow_down: | | [src/derivatives/linalg/arithmetic.jl](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2Rlcml2YXRpdmVzL2xpbmFsZy9hcml0aG1ldGljLmps) | `69.67% <0.00%> (-1.45%)` | :arrow_down: | | ... and [10 more](https://codecov.io/gh/JuliaDiff/ReverseDiff.jl/pull/209/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[devmotion]

Bump :slightly_smiling_face:

It would be good to fix ReverseDiff such that we can move forward with https://github.com/JuliaDiff/DiffRules.jl/pull/79.

[mohamed82008]

Sorry for the delay, been swamped recently. I will take a look tonight.

[mohamed82008]

Ah the methods are defined a bit further in the same file, nevermind.