Automatic CI testing Torch backend on GPU

[dsyme]

@pkese has done some great work testing the Torch backend with GPU enabled.

See discussion from https://github.com/xamarin/TorchSharp/issues/143#issuecomment-637669892

Let's continue the discussion here since it's about DiffSharp not TorchSharp now

[dsyme]

@pkese reports that all tests now pass!

https://github.com/DiffSharp/DiffSharp/pull/121#issuecomment-637740358

We will now need to automate the testing both in-repo and during CI

[dsyme]

Just to note that Torch testing on GPU could in theory be automated in Azure DevOps using GPU-enabled containers. However

• We need a manual container group in the devops, and this costs money

• CUDA 9.0 is supported (not 10.2 as currently supported by TorchSharp), so we'd new libtorch-cuda-9.0 packages

• Linux only

• The images in the container group may take 8-10 minuttes to spin up I think

https://docs.microsoft.com/en-us/azure/container-instances/container-instances-gpu

[VisualMelon]

Would a PR along the lines of https://github.com/DiffSharp/DiffSharp/compare/dev...VisualMelon:GpuTest be welcome? It makes it easier to test with GPU because you don't need to make any code change (I can set DIFFSHARP_TESTGPU=true for my user and it will always test with GPU). Something like this would be necessary for testing on a dedicated CI. I'm not attached to DIFFSHARP_TESTGPU if others don't like the name. (Note: I can't test GPU on linux at the moment)

[dsyme]

Yes that looks good

Maybe add to DEVGUIDE.md the single-line invocation of dotnet test providing the property on the command line for cut-and-paste

[VisualMelon]

Done. I'll open a PR.