Feature Request: Faster Analysis (GPU Based) or even Live Analysis.

[destinrl]

Putting this in per cirquit's request. Basically, I would love it if beefier machines could somehow lend GPU power to the analysis phase. If we can speed up analysis of a video file that would be amazing. If you could even have a basic FPS analysis running live while doing capture that would be phenomenal.

Not sure how Nvidia or Digital Foundry does it, but maybe it's possible. Being able to capture data live could output a csv file that could then be ingested in TrDrop or even Premiere to create the overlay of your choice as an image sequence or .mov with alpha. That level of development seems like it would be years out though for a solo project.

[cirquit]

So I see a few issues with this feature

• making a "live" analysis software is basically impossible as of right now as this is a whole other project that works with streams of videos (think gstreamer plugin, which can make video pipelines), compared to trdrop that needs video file(s) right now
• using the GPU to enhance the performance is a very obvious idea, but I'm not sure if OpenCV (the library I'm using to do the image comparison work) can get more performant than how it already is. I already parallelized the comparison for each video so its multithreaded by default (1 thread per video), but it might be a good idea to check where the performance bottlenecks are
• The currently biggest problem is the encoding of the images and writing them to disk. I feel the IO is the thing that's blocking further performance increases, especially when I increase the resolution. But this may be also a false hint, I think I have to check out some profiling tools of Qt and see what insights they can provide

What I don't want Trdrop to be is a program that ingests .csvs and creates the overlay, that should be easily done in Premiere by a script.

What we can talk about is using the algorithms (if you can even call them that) in trdrop to make a plugin for other programs (I think you mentioned one in one of your emails) to enable the live capture analysis.

[dubigrasu]

What I don't want Trdrop to be is a program that ingests .csvs and creates the overlay, that should be easily done in Premiere by a script.

That's so unfortunate. I've been following trdrop developments for a while, and that's exactly the feature that I hoped it will be implemented one day.

[cirquit]

What I don't want Trdrop to be is a program that ingests .csvs and creates the overlay, that should be easily done in Premiere by a script.

That's so unfortunate. I've been following trdrop developments for a while, and that's exactly the feature that I hoped it will be implemented one day.

The reason for that is quite simple - trdrop is for analysis purposes mainly, not for being the best at visualization. I literally create all the plots programatically, which is a big pain in the ass when handling different resolutions.

Adobe Premerie is specifically designed to apply visual effects to videos, and does that better by a very far margin compared to what trdrop provides. Creating a similar overlay to what trdrop does should take at max 1h for somebody who knows Premiere. Ingesting the csv which is either created by trdrop/any other analysis program, is also easily done in Premiere.

The reason I made trdrop was to provide an easy way for peope who want a complete solution for analysis combined with added visualizations to help with the analysis. Not to provide the best possible visualization on par with Premiere or other professional video editing software.

[dubigrasu]

Well, I didn't wished for best visualization on par with Premiere, just an integrated basic feature/tool for people that don't know or own Premiere (like me). I think that would be closer to an actual complete solution.

To be clear, what I wanted to see is the possibility to feed trdrop a .csv file of my own and obtain an overlay (video or sequence) that I could use later. I imagined that since trdrop is already able to do the overlay, it should be doable to make it work with my csv/data (instead of the data that it extracts itself).

But since this feature is not gonna be added, is there any hackish/way/workaround to accomplish that? (like substitute/replace the trdrop data with my own somewhere in the middle of the process...I dunno, something)

[cirquit]

So I understand the base argument, but it's unfortunately not as easy as it might seem. Pipelining the data from a .csv into the "layout" makes a few other things unusable (like all the pixel-difference settings/delta rendering) so it would be a whole different "mode" of trdrop, which is not analyzing at all. This would mean disabling multiple settings which is (I know how dumb it sounds) kind of hard in QML, as there are some major issues about how this works.

The reason why I'm still against that is that I do not want people to have the chance to fake recordings with this software. I know it's still possible, but I do not want to enable this further. If people would have an incentive to fake framerates, this would easily be done by changing the .csv values manually and importing them into trdrop. By creating this barrier of not supporting this, and forcing people to either create their own overlay with Premiere or other similar tools, makes it harder for people to even try this.

Think of trdrop as a watermarking tool (which we are far far away, unfortunately, but still), that creates an overlay that can not be messed with outside of the configuration parameters. It's still possible to create slightly better results by tweaking the pixel-difference option, but I hope to forcefully include this option on the screen sometime so it's open and clear for everybody.

I think there are no possibilities to do that besides getting to know one of the software solutions for video editing. Take a look at this list, I've heard good things about Davinci Resolve.

[dubigrasu]

The reason why I'm still against that is that I do not want people to have the chance to fake recordings with this software.

Hmm, I understand now. By not having this "feature" trdrop aims to guarantee that the analysis is honest and reliable, and not a result of possible interference due to good/bad intentions or just unqualified data manipulations.

Rest assured that I wasn't planning to fake results, but I see what you mean, and you are right. Thanks for the explanations, and trdrop of course.

[destinrl]

I think what I had in mind was just speeding up the analysis as it currently is, or adding the ability to analyze live during capture and then apply the plotting in premiere somehow using the new tools Adobe has added such as the Essential Graphics. Just a thought!

[illusion0001]

In progress using obs and lvk. https://github.com/Crowsinc/LiveVisionKit/compare/main...illusion0001:LiveVisionKit:fps-view https://youtu.be/05XvbIb4vqg https://youtu.be/c3Iciu0ylgo

[illusion0001]

@cirquit might be interested in this new solution, but I still am having trouble on how to calculate frame rate from it. Also found that gray scale is quite limiting and works best for vsync videos, will have to investigate a better solution for tearing.