search

OpenSourcePhysics / tracker

Video analysis and modeling tool built on the Open Source Physics framework
GNU General Public License v3.0
206 stars 52 forks source link

GPU Acceleration #32

Open kevtan opened 3 years ago

kevtan commented 3 years ago

Hi, I've been using Tracker for my mechanics lab course at school and it's a really useful piece of software! It doesn't seem to be very performant on the high-resolution videos that modern smartphones are able to put out, though. I was wondering if there's been any work (or interest) in using the GPU to accelerate some of the computations being performed? I took at brief look at the code and there doesn't seem to be any support for GPU acceleration. It has the potential to make Tracker dramatically faster!

dobrown commented 3 years ago

Hi Kevin, I don't know anything about adding GPU acceleration to Tracker but if you do I'd like to know more. Are you aware of the beta JavaScript Tracker at https://tracker.physlets.org/trackerJS/ ? Could GPU acceleration apply in JS as well? Doug

On Sat, Nov 14, 2020 at 10:01 AM Kevin Tan notifications@github.com wrote:

Hi, I've been using Tracker for my mechanics lab course at school and it's a really useful piece of software! It doesn't seem to be very performant on the high-resolution videos that modern smartphones are able to put out, though. I was wondering if there's been any work (or interest) in using the GPU to accelerate some of the computations being performed? I took at brief look at the code and there doesn't seem to be any support for GPU acceleration. It has the potential to make Tracker dramatically faster!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OpenSourcePhysics/tracker/issues/32, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXIBFDFCLOCUF6HZLTHW7DSP3AXTANCNFSM4TVVVLYQ .

kevtan commented 3 years ago

Hi Doug, sorry for the late response! I'm not aware of the beta JavaScript Tracker—thanks for bringing that to my attending. I did some research and it looks like GPU acceleration would apply to JavaScript as well: GPU.js. It looks like there are GPU interfaces for Java as well (for the stable version of Java).

Since I'm not as familiar as you with the inner workings of Tracker, I was wondering if you might be able to take a look at this Stack Overflow post that talks about using Java to access the compute power of the GPU and what the GPU is actually good for.

In particular, does Tracker have a lot of data-parallelism where the same computation is done on many different parts of an image? At first blush, it looks like this happens in the current iteration of the auto-tracking feature where the cross-correlation is computed between a template image and different patches of an image to check for a match. I might not have completely understood the code, though. Do you know of any other parts that exhibit this kind of data parallelism?

dobrown commented 3 years ago

Hi Kevin, thanks for the link to the GPU info--really great information there!

Yes, I do think the autotracking computations could benefit from GPU processing. As you say, the template is compared with all patches within a search area. So you need to load the search area pixels and template pixels, then do the identical (multiple) computations for all position indices. The current code definitely slows down significantly when the search area and/or template get big. If you want to look at the source code, see org.opensourcephysics.media.core.TemplateMatcher.getMatchLocation(BufferedImage target, Rectangle searchRect)

I can't think of any other Tracker code that would benefit enough from GPU to make it worthwhile..

Interesting stuff--thanks! Doug

On Thu, Nov 26, 2020 at 1:56 PM Kevin Tan notifications@github.com wrote:

Hi Doug, sorry for the late response! I'm not aware of the beta JavaScript Tracker—thanks for bringing that to my attending. I did some research and it looks like GPU acceleration would apply to JavaScript as well: GPU.js https://gpu.rocks/#/. It looks like there are GPU interfaces for Java as well (for the stable version of Java).

Since I'm not as familiar as you with the inner workings of Tracker, I was wondering if you might be able to take a look at this https://stackoverflow.com/questions/22866901/using-java-with-nvidia-gpus-cuda Stack Overflow post that talks about using Java to access the compute power of the GPU and what the GPU is actually good for.

In particular, does Tracker have a lot of data-parallelism where the same computation is done on many different parts of an image? At first blush, it looks like this happens in the current iteration of the auto-tracking feature where the cross-correlation is computed between a template image and different patches of an image to check for a match. I might not have completely understood the code, though. Do you know of any other parts that exhibit this kind of data parallelism?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OpenSourcePhysics/tracker/issues/32#issuecomment-734490059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXIBFHEUN5UWGSSU2BM3H3SR3FJPANCNFSM4TVVVLYQ .