search

mlverse / torchvision

R interface to torchvision
https://torchvision.mlverse.org
Other
62 stars 15 forks source link

torchvision.ops #56

Open dominikj2 opened 3 years ago

dominikj2 commented 3 years ago

Hi @dfalbel ,

Is it possible to have an R Torch version of torchvision.ops (operators for Computer Vision).

https://pytorch.org/vision/stable/ops.html

p.s. I am particularly interested in torchvision.ops.roi_align.

I'd be interested in learning to help get these operators available in R Torch if provided with some guidance. I have python and R programming background but no C++. Willing to learn C++ on the task if its a small set of tasks that needs repetition for each operator. Not sure if my proposition sounds helpful.

dfalbel commented 3 years ago

Currently this is not possible. This is something I'll be working on after the next release of torch.

I will let you know hot it goes, but I think at least the first iteration to implement this is quite complicated. I don't have yet clear on my mind how this should work.

To unblock you, one thing that you could do is to add the code from roi_align into torch. The approach would be similar to: https://github.com/mlverse/torch/pull/619/files

dominikj2 commented 3 years ago

Yes, you did https://github.com/mlverse/torch/pull/619/files for me and I thought that integrating roi_align could be implemented with a similar approach.

Do you have resources that I could learn from to apply this approach? I would like to start learning and hopefully contribute more. Maybe (if it's not a hassle for you) next time you do something similar to https://github.com/mlverse/torch/pull/619/files, I would love to do a zoom meeting with recorded screen share showing the process? Only if it doesn't slow you down. I'm not sure how much of the process follows a "template" and "standard procedure" and how much of it is very unique to each torchvision.op. Anyway keep me in mind as I want to learn and contribute.

dominikj2 commented 3 years ago

Hi @dfalbel,

I spent much of the day learning JIT, LibTorch distribution, and Torch Script ( tracing and annotation). Well done with JIT tracing for Torch! Is you next plan to develop ScriptModule for control flow capabilities and more (i.e. torch.jit.script) ?

Quick question, when R Torch has torch.jit.script capabilities, will that mean all/most pytorch torchvision.op can be converted to Torch Script and loaded into R Torch using JIT? Would this effectively automate the procedure that you undertook to put sort_vertices in contrib(i.e. building C++ application using CMake and LibTorch for the contrib.op)?