Add CUDA derived global kernels' compilation support in reverse mode

[kchristin22]

This PR solves #1036. CUDA kernels can now be an arg to a clad::gradient function and the derived kernels can be executed successfully.

Details on issues faced and the final approach can be found here.

[parth-07]

@kchristin22 Can you please explain the solution/design as part of the pull-request description?

[kchristin22]

@kchristin22 Can you please explain the solution/design as part of the pull-request description?

Yes, sorry for delaying this. I added a comment in the issue to facilitate further questions and discussions and added a link to the comment in the description of this PR.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 75.86207% with 14 lines in your changes missing coverage. Please review.

Project coverage is 93.96%. Comparing base (6cc83ee) to head (fd80274). Report is 17 commits behind head on master.

Files	Patch %	Lines
lib/Differentiator/VisitorBase.cpp	30.76%	9 Missing :warning:
lib/Differentiator/DiffPlanner.cpp	66.66%	4 Missing :warning:
include/clad/Differentiator/Differentiator.h	94.11%	1 Missing :warning:

Additional details and impacted files

[](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) ```diff @@ Coverage Diff @@ ## master #1041 +/- ## ========================================== - Coverage 94.09% 93.96% -0.14% ========================================== Files 55 55 Lines 8248 8296 +48 ========================================== + Hits 7761 7795 +34 - Misses 487 501 +14 ``` | [Files](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) | Coverage Δ | | |---|---|---| | [include/clad/Differentiator/DerivativeBuilder.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDerivativeBuilder.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0Rlcml2YXRpdmVCdWlsZGVyLmg=) | `100.00% <ø> (ø)` | | | [include/clad/Differentiator/DiffPlanner.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDiffPlanner.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0RpZmZQbGFubmVyLmg=) | `69.56% <ø> (ø)` | | | [lib/Differentiator/BaseForwardModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FBaseForwardModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0Jhc2VGb3J3YXJkTW9kZVZpc2l0b3IuY3Bw) | `98.53% <100.00%> (ø)` | | | [lib/Differentiator/DerivativeBuilder.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FDerivativeBuilder.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0Rlcml2YXRpdmVCdWlsZGVyLmNwcA==) | `100.00% <100.00%> (ø)` | | | [lib/Differentiator/HessianModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FHessianModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0hlc3NpYW5Nb2RlVmlzaXRvci5jcHA=) | `99.53% <100.00%> (ø)` | | | [lib/Differentiator/ReverseModeForwPassVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FReverseModeForwPassVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1JldmVyc2VNb2RlRm9yd1Bhc3NWaXNpdG9yLmNwcA==) | `98.76% <100.00%> (-0.66%)` | :arrow_down: | | [lib/Differentiator/ReverseModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FReverseModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1JldmVyc2VNb2RlVmlzaXRvci5jcHA=) | `97.77% <100.00%> (+0.02%)` | :arrow_up: | | [lib/Differentiator/VectorForwardModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FVectorForwardModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1ZlY3RvckZvcndhcmRNb2RlVmlzaXRvci5jcHA=) | `100.00% <100.00%> (ø)` | | | [include/clad/Differentiator/Differentiator.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDifferentiator.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0RpZmZlcmVudGlhdG9yLmg=) | `80.85% <94.11%> (-2.49%)` | :arrow_down: | | [lib/Differentiator/DiffPlanner.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FDiffPlanner.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0RpZmZQbGFubmVyLmNwcA==) | `97.79% <66.66%> (-0.95%)` | :arrow_down: | | ... and [1 more](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) | | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/vgvassilev/clad/pull/1041/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) | [Files](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) | Coverage Δ | | |---|---|---| | [include/clad/Differentiator/DerivativeBuilder.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDerivativeBuilder.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0Rlcml2YXRpdmVCdWlsZGVyLmg=) | `100.00% <ø> (ø)` | | | [include/clad/Differentiator/DiffPlanner.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDiffPlanner.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0RpZmZQbGFubmVyLmg=) | `69.56% <ø> (ø)` | | | [lib/Differentiator/BaseForwardModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FBaseForwardModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0Jhc2VGb3J3YXJkTW9kZVZpc2l0b3IuY3Bw) | `98.53% <100.00%> (ø)` | | | [lib/Differentiator/DerivativeBuilder.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FDerivativeBuilder.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0Rlcml2YXRpdmVCdWlsZGVyLmNwcA==) | `100.00% <100.00%> (ø)` | | | [lib/Differentiator/HessianModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FHessianModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0hlc3NpYW5Nb2RlVmlzaXRvci5jcHA=) | `99.53% <100.00%> (ø)` | | | [lib/Differentiator/ReverseModeForwPassVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FReverseModeForwPassVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1JldmVyc2VNb2RlRm9yd1Bhc3NWaXNpdG9yLmNwcA==) | `98.76% <100.00%> (-0.66%)` | :arrow_down: | | [lib/Differentiator/ReverseModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FReverseModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1JldmVyc2VNb2RlVmlzaXRvci5jcHA=) | `97.77% <100.00%> (+0.02%)` | :arrow_up: | | [lib/Differentiator/VectorForwardModeVisitor.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FVectorForwardModeVisitor.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL1ZlY3RvckZvcndhcmRNb2RlVmlzaXRvci5jcHA=) | `100.00% <100.00%> (ø)` | | | [include/clad/Differentiator/Differentiator.h](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=include%2Fclad%2FDifferentiator%2FDifferentiator.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-aW5jbHVkZS9jbGFkL0RpZmZlcmVudGlhdG9yL0RpZmZlcmVudGlhdG9yLmg=) | `80.85% <94.11%> (-2.49%)` | :arrow_down: | | [lib/Differentiator/DiffPlanner.cpp](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree&filepath=lib%2FDifferentiator%2FDiffPlanner.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev#diff-bGliL0RpZmZlcmVudGlhdG9yL0RpZmZQbGFubmVyLmNwcA==) | `97.79% <66.66%> (-0.95%)` | :arrow_down: | | ... and [1 more](https://app.codecov.io/gh/vgvassilev/clad/pull/1041?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev) | | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/vgvassilev/clad/pull/1041/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Vassil+Vassilev)

[vgvassilev]

@kchristin22, what happened here? Something to do with rebase/git?

[kchristin22]

@vgvassilev I opened a new one to have a clean slate. The old branch is renamed and kept in my repository in case it's ever needed.

[vgvassilev]

Ok, but please do not generally do that since we lose review history. Next time just ping me and I can help you.

[kchristin22]

@vgvassilev ok, sorry for that. There weren't many comments here, so I thought it would be ok. I'll ping you next time.