WindowTop / WindowTop-App

Set window on top, make it dark, transparent and more
Other
1.16k stars 69 forks source link

[Glass & Dark Mode] Improve rendering performances methods #105

Closed gileli121 closed 1 year ago

gileli121 commented 3 years ago

Currently, the dark mode feature needs to process the frame of the window, and using an algorithm it assumes where exactly are the images in the frame. Then it knows to not invert the colors in these areas.

Same for the Glass mode - The glass mode needs also another algorithm that detects any text in the frame.

After more investigation of the available APIs in the system, seems that these algorithms are unnecessary. The system has API to get directly the DOM of any window

This API allows you to get directly all of the information that is needed. Now it is possible to know where exactly the window showing an image, where it shows text and without any need to recognize it using an algorithm that try to figure it out using just pixels.

Tasks:

This should fix the issues:

gileli121 commented 2 years ago

I found a new way how to improve the rendering performances! Until a few days ago, I was unaware of how to invert the colors of 8,294,400 pixels (the amount in 4K resolution) in GPU without using NVIDIA CUDA API (which requires NVIDIA GPU).

And recently, I found a way to do it! In Dricent2D API, there is a GPU-accelerated method to do that, and because it is Dricent2D, it is not exclusive to NVIDIA GPUs! It will work on even regular Intel GPUs. This finding means that every computer will use the GPU for this effect. Great stuff!

I made a POC that proves that it is possible to get 60+ frames per second on 4K using just intel GPU, and this is instead of 20-30 FPS.

Not only that, but this finding also comes with a new optimization advantage: because the OS API also does not send the captured frame to the CPU and because Dricent2D will do its magic in GPU, now, unlike before, we will never need to move any frame data to CPU (unless the "filter images" option is enabled) so this fact also will help. This will also reduce a lot of battery usage because everything will be done natively by the GPU!

The idea in the message above (To use DOM API) may still be worth checking after this finding. It is still relevant because it tries to optimize the performances when the "filter images" option is enabled.

Opened task #264 for this optimization method