Black screen when capturing from multiple clients

[Kuhicop]

I had a game bot that worked by image recognition and I'm trying to use Direct3DHook so it can detect images in background

The problem is that sometimes I'm getting black screens as result:

I guess the bot is trying to process images before they're captured.

I don't understand why processing images is an async method while processing them takes only a few miliseconds.

For example, trying to do: CaptureImage(); Thread.Sleep(1000); It will only do the Sleep but not the CaptureImage();

[justinstenning]

So just sometimes it is a black screen when normally it works fine? Generally a black screen means that something in the present hook is either wrong, or something else around D3D is the cause (Eg targeting incorrect version etc, however that would not be intermittent). I don’t think this is a threading issue.

Where is CaptureImage? Is that yours? The implementation does use a blocking approach see CaptureInterface.GetScreenshot vs BeginGetScreenshot, so unless you are not using that it should be fine in that regard.

But yes underneath it is all async because you are capturing the image using the GPU, which is not in step with the CPU. Depending on how you are using it it is also talking across processes. But this is handled for you.

[Kuhicop]

I'm adding an example of my application.

If you check at timerLoop_Tick first I want to capture the game screen and after doing the rest.

If I do like in the example, it won't work because it will ignore CaptureImage(); and will jump to the next instructions.

If I do it reverse, checking if (bmp != null) { //do something } then as last instruction CaptureImage() it will work, but sometimes I will get a black screen.

Bitmap bmp;

private void Form1_Load(object sender, EventArgs e)
{
    InjectToClient();
}

private void timerLoop_Tick(object sender, EventArgs e)
{
    CaptureImage();
    // Check if a given image is in the game taken screenshot by Direct3DHook (bmp)
}

int processId = 0;
Process _process;
CaptureProcess _captureProcess;
private bool AttachProcess()
{
    // Skip if the process is already hooked (and we want to hook multiple applications)
    if (HookManager.IsHooked(process.Id))
    {
        return false;
    }

    Direct3DVersion direct3DVersion = Direct3DVersion.AutoDetect;

    CaptureConfig cc = new CaptureConfig()
    {
        Direct3DVersion = direct3DVersion,
        ShowOverlay = false
    };

    processId = process.Id;
    _process = process;

    var captureInterface = new CaptureInterface();
    captureInterface.RemoteMessage += new MessageReceivedEvent(CaptureInterface_RemoteMessage);
    _captureProcess = new CaptureProcess(process, cc, captureInterface);

    Thread.Sleep(10);

    if (_captureProcess == null)
    {
        WriteLog("No executable found");
    }

    return true;
}

/// <summary>
/// Display messages from the target process
/// </summary>
/// <param name="message"></param>
void CaptureInterface_RemoteMessage(MessageReceivedEventArgs message)
{
    txtDebugLog.Invoke(new MethodInvoker(delegate ()
        {
            txtDebugLog.Text = String.Format("{0}\r\n{1}", message, txtDebugLog.Text);
        })
    );
}

/// <summary>
/// Display debug messages from the target process
/// </summary>
/// <param name="clientPID"></param>
/// <param name="message"></param>
void ScreenshotManager_OnScreenshotDebugMessage(int clientPID, string message)
{
    txtDebugLog.Invoke(new MethodInvoker(delegate ()
        {
            txtDebugLog.Text = String.Format("{0}:{1}\r\n{2}", clientPID, message, txtDebugLog.Text);
        })
    );
}

DateTime start;

private void CaptureImage()
{
    start = DateTime.Now;
    progressBar1.Maximum = 1;
    progressBar1.Step = 1;
    progressBar1.Value = 0;

    DoRequest();
}

/// <summary>
/// Create the screen shot request
/// </summary>
void DoRequest()
{
    try
    {
        progressBar1.Invoke(new MethodInvoker(delegate ()
        {
            if (progressBar1.Value < progressBar1.Maximum)
            {
                progressBar1.PerformStep();

                Size? resize = null;
                _captureProcess.CaptureInterface.BeginGetScreenshot(new Rectangle(0, 0, 0, 0), new TimeSpan(0, 0, 2), Callback, resize, (ImageFormat)Enum.Parse(typeof(ImageFormat), "Bitmap"));
            }            
        })
        );
    }
    catch
    {

    }
}

/// <summary>
/// The callback for when the screenshot has been taken
/// </summary>
/// <param name="clientPID"></param>
/// <param name="status"></param>
/// <param name="screenshotResponse"></param>
void Callback(IAsyncResult result)
{
    Screenshot screenshot;
    try
    {
        screenshot = _captureProcess.CaptureInterface.EndGetScreenshot(result);
    }
    catch
    {
        screenshot = null;
    }

    if (screenshot != null)
    {
        using (screenshot)
            try
            {
                _captureProcess.CaptureInterface.DisplayInGameText("Screenshot captured...");
                if (screenshot != null && screenshot.Data != null)
                {
                    bmp = new Bitmap(screenshot.ToBitmap());
                }

                Thread t = new Thread(new ThreadStart(DoRequest));
                t.Start();
            }
            catch
            {
            }
        while (!result.IsCompleted)
        {
            Thread.Sleep(200);
        }
    }
}

[justinstenning]

Your code is async (using BeginGetScreenshot), the image is not necessarily captured immediately after CaptureImage. Either use some thread sync, or using the blocking GetScreenshot.

[Kuhicop]

So, in your TestScreenshot example, I just changed the DoRequest() method like this:

Size? resize = null;
if (!string.IsNullOrEmpty(txtResizeHeight.Text) && !String.IsNullOrEmpty(txtResizeWidth.Text))
        resize = new System.Drawing.Size(int.Parse(txtResizeWidth.Text), int.Parse(txtResizeHeight.Text));
var res = _captureProcess.CaptureInterface.GetScreenshot(new Rectangle(int.Parse(txtCaptureX.Text), int.Parse(txtCaptureY.Text), int.Parse(txtCaptureWidth.Text), int.Parse(txtCaptureHeight.Text)),
                                                                       new TimeSpan(0, 0, 2),
                                                                       resize,
                                                                       (ImageFormat)Enum.Parse(typeof(ImageFormat), cmbFormat.Text)
                                                                       );
pictureBox1.Image = res.ToBitmap();

But I'm getting error at ScreenshotExtensions.cs because it appears that screenshot is null

Could you help me please? I don't know what I'm missing

[justinstenning]

Going back a step, does it work with TestScreenshot at all, unchanged? If not then it could be a range of issues on the injected side (ie with the D3D code not being compatible with the target etc etc.

[Kuhicop]

Yeah it's working perfectly with TestScreenshot unchanged

[Kuhicop]

Not being able to make this work: GetScreenshot

I've just tested now by replacing the btnCapture_Click event code with var res = _captureProcess.CaptureInterface.GetScreenshot();

I get null as return

[Aks-4125]

I am also getting null at _captureProcess.CaptureInterface.GetScreenshot();