Highly configurable and responsive World of Warcraft Classic pixel Grind Bot - No DLL injection or memory tampering, just screen capture and input simulation.
177
stars
121
forks
source link
Refactor: Game: Core: SharedLib: While `WoWScreenDXGI` is used, every image processing sourced from `DXGI` instead of using some part of `GDI` #547
From now on when DXGI reader type is used, every part of the application sources the Bitmap data respectively.
GDI based Graphics.CopyFromScreen can be called anytime anywhere, however the downside is that, its going to block the thread for a pretty long time around ~20-30ms. During this time possible one or two frames might get lost while running the game in high FPS scenario.
DXGI based approach however can produce frame data around roughly every 10ms and AcquireNextFrame has the option to don't await for the next frame. Copying the bitmap data from the GPU to the system memory takes around 1-2ms(at least on my test machine), however this can be significantly reduced by using shaders in the future.
DXGI has one big limitation, there can be only instance of IDXGIOutputDuplication so the previous setup wasn't ideal.
Where every major bitmap extracting process had its own Thread(AddonThread, ScreenShotThread, MinimapThread) all these logic had to merged together. WoWScreenDXGI has exclusive access to IDXGIOutputDuplication. When a new frame is present its going to copy the bitmap data from the GPU memory to system memory. This can be really taxing, so instead only the AddonData is obtained every frame, and the whole Screen and Minimap only acquired with a discrete intervals this way the CPU usage kept low as possible.
There was one major problem with the current setup. WoWScreenDXGI acts as IAddonDataProvider and IWoWScreen as well. So two or more threads are now fighting for the same Bitmap resource.
GDI Bitmap has one downside, at a singular moment, there can be one and only thread access to the Bitmap. Sadly the ReaderWriterLockSlim doesn't help at all. During this time even accessing any kind of property for reading will throw 'object used in elsewhere' exception. LockBits out of the question. Each place where the Bitmap could be accessed, have to acquire the lock first, including the BlazorServer Frontend, Core.ScreenCapture etc...
After many trial and errors, managed to come up a solution which only produce a really small amount of code duplication in the BotController, however maximises the CPU utilization as much as possible but keeps CPU usage also low compare to the GDI solution while using DXGI.
While NpcNameFinder is enabled the CPU usage should remain as low as possible.
The first major benefit which DXGI brings is that, the Game client window pixel data is now available to the program. On the other hand opens up the possibility to migrate NpcNameFinder and MinimapNodeFinder image processing algorithms to the GPU probably as Compute Shaders.
Secondly it has a higher chance to avoid frame skipping. Not to mention, it could lower the latency between actions. However this needs to be balanced, it has a trade off between CPU time and sleep.
From CoreTests project the NpcNameOverlay has been pulled into Core which means now HeadlessServer and BlazorServer has the possibility during runtime to visualize the NpcNameFinder state on the screen with an Overlay, this can be handy for debugging purposes. Can be enabled by CLI or editing the appsettings.json.
Also fixed an index by one issue in the NpcNameFinder after filtering out the Empty Rects.
From now on when
DXGIreader type is used, every part of the application sources the Bitmap data respectively.GDIbasedGraphics.CopyFromScreencan be called anytime anywhere, however the downside is that, its going to block the thread for a pretty long time around ~20-30ms. During this time possible one or two frames might get lost while running the game in high FPS scenario.DXGIbased approach however can produce frame data around roughly every 10ms and AcquireNextFrame has the option to don't await for the next frame. Copying the bitmap data from the GPU to the system memory takes around 1-2ms(at least on my test machine), however this can be significantly reduced by using shaders in the future.DXGIhas one big limitation, there can be only instance of IDXGIOutputDuplication so the previous setup wasn't ideal.Where every major bitmap extracting process had its own Thread(AddonThread, ScreenShotThread, MinimapThread) all these logic had to merged together.
WoWScreenDXGIhas exclusive access toIDXGIOutputDuplication. When a new frame is present its going to copy the bitmap data from the GPU memory to system memory. This can be really taxing, so instead only the AddonData is obtained every frame, and the whole Screen and Minimap only acquired with a discrete intervals this way the CPU usage kept low as possible.There was one major problem with the current setup.
WoWScreenDXGIacts asIAddonDataProviderandIWoWScreenas well. So two or more threads are now fighting for the same Bitmap resource.GDI Bitmap has one downside, at a singular moment, there can be one and only thread access to the Bitmap. Sadly the ReaderWriterLockSlim doesn't help at all. During this time even accessing any kind of property for reading will throw 'object used in elsewhere' exception. LockBits out of the question. Each place where the Bitmap could be accessed, have to acquire the lock first, including the BlazorServer Frontend,
Core.ScreenCaptureetc...After many trial and errors, managed to come up a solution which only produce a really small amount of code duplication in the
BotController, however maximises the CPU utilization as much as possible but keeps CPU usage also low compare to theGDIsolution while usingDXGI.While
NpcNameFinderis enabled the CPU usage should remain as low as possible.The first major benefit which
DXGIbrings is that, the Game client window pixel data is now available to the program. On the other hand opens up the possibility to migrateNpcNameFinderandMinimapNodeFinderimage processing algorithms to the GPU probably as Compute Shaders.Secondly it has a higher chance to avoid frame skipping. Not to mention, it could lower the latency between actions. However this needs to be balanced, it has a trade off between CPU time and sleep.
From
CoreTestsproject theNpcNameOverlayhas been pulled intoCorewhich means nowHeadlessServerandBlazorServerhas the possibility during runtime to visualize theNpcNameFinderstate on the screen with an Overlay, this can be handy for debugging purposes. Can be enabled by CLI or editing theappsettings.json.Also fixed an index by one issue in the NpcNameFinder after filtering out the Empty Rects.