search

BoboTiG / python-mss

An ultra fast cross-platform multiple screenshots module in pure Python using ctypes.
https://pypi.org/project/mss/
MIT License
1.04k stars 94 forks source link

ScreenShotError: gdi32.GetDIBits() failed. #267

Open ColdLighting opened 1 year ago

ColdLighting commented 1 year ago

General information:

Description of the warning/error

I packaged my project as an exe using pyinstaller and ran it. At some point on the 6th day, I reported a ScreenShotError error when taking a screenshot, and an AttributeError error occurred in subsequent screenshots.

Full message

ERROR 2023/09/13 22:06:25 [pool_exception_util.py:18] ：
Traceback (most recent call last):
  File "utils\pool_exception_util.py", line 15, in __call__
  File "core\mouse_monitor.py", line 229, in async_do_click
  File "core\mouse_monitor.py", line 266, in save_fs_img
  File "mss\base.py", line 90, in grab
  File "mss\windows.py", line 252, in _grab_impl
mss.exception.ScreenShotError: gdi32.GetDIBits() failed.

ERROR 2023/09/13 22:06:36 [pool_exception_util.py:18] ：
Traceback (most recent call last):
  File "utils\pool_exception_util.py", line 15, in __call__
  File "core\mouse_monitor.py", line 229, in async_do_click
  File "core\mouse_monitor.py", line 266, in save_fs_img
  File "mss\base.py", line 90, in grab
  File "mss\windows.py", line 250, in _grab_impl
AttributeError: '_thread._local' object has no attribute 'data'

ERROR 2023/09/13 22:06:37 [pool_exception_util.py:18] ：
Traceback (most recent call last):
  File "utils\pool_exception_util.py", line 15, in __call__
  File "core\mouse_monitor.py", line 229, in async_do_click
  File "core\mouse_monitor.py", line 266, in save_fs_img
  File "mss\base.py", line 90, in grab
  File "mss\windows.py", line 250, in _grab_impl
AttributeError: '_thread._local' object has no attribute 'data'

and so on ...

Other details

First of all, thank you very much for providing a high-performance screenshot tool, I like it very much!

The following is a detailed description and part of the code for the problem I encountered.

I used pynput.mouse to monitor the mouse. When the mouse is left-clicked on the specified area in the program window I specified, I will take a screenshot and save some other additional coordinate information. Then I used pyinstaller to package my project as an exe and run it, but after about 6 days of running it threw a ScreenShotError at some point, and all subsequent clicks would throw an exception AttributeError: _data, and subsequently I obtained the system zoom and foreground window coordinates. , the xy value passed by pynput will become 0 !

I checked the relevant issues and seemed to not find a solution to the problem related to me. Then I tried to read your source code and found that the location where the exception was thrown was in mss/windows. py:252: 

gdi.BitBlt(memdc, 0, 0, width, height, srcdc, monitor["left"], monitor["top"], SRCCOPY | CAPTUREBLT)
bits = gdi.GetDIBits(memdc, self._handles.bmp, 0, height, self._handles.data, self._handles.bmi, DIB_RGB_COLORS)
if bits != height:
    raise ScreenShotError("gdi32.GetDIBits() failed.")

it judged bits != height. I checked my logs. When the last error was reported, the monitor parameter I passed was correct: (3, 22, 794, 629)

INFO 2023/09/13 22:06:25 [mouse_monitor.py:120] origin window rect：(3, 22, 794, 629)
INFO 2023/09/13 22:06:25 [mouse_monitor.py:128] sys scaling: 1.0
INFO 2023/09/13 22:06:25 [mouse_monitor.py:146] di area：min：(290,34) max：(769,290) click coord(relative client)：(332,287)
INFO 2023/09/13 22:06:25 [mouse_monitor.py:159] clicked in di area, start processing
INFO 2023/09/13 22:06:25 [mouse_monitor.py:228] save screenshot
INFO 2023/09/13 22:06:25 [mouse_monitor.py:261] screenshot coord：[3, 22, 794, 629]
WARNING 2023/09/13 22:06:25 [pool_exception_util.py:17] An error occurred, see error.log

so it can only be the value of bits has a problem. But sorry, I don't know much about win32api, so I went to Microsoft's official website to check the relevant documents and roughly understood the role of BitBlt/GetDIBits/GetWindowDC. I guess the root cause may be that an invalid handle was obtained when windows.py:112 was initialized causing this error: 

 self._handles.srcdc = self.user32.GetWindowDC(0)

I briefly checked the code of pynput.mouse and learned that it is a single-thread processing callback function, which means that the first time this error occurred in its internal sub-thread, it affected all my subsequent code that calls win32api, just like this:

INFO 2023/09/13 22:06:27 [mouse_monitor.py:120] origin window rect：(0, 0, 0, 0)
INFO 2023/09/13 22:06:27 [mouse_monitor.py:128] sys scaling: 0.0
INFO 2023/09/13 22:06:27 [mouse_monitor.py:146] di area：min：(0, 0) max：(0, 0) click coord(relative client)：(0, 0)
INFO 2023/09/13 22:06:27 [mouse_monitor.py:159] clicked in di area, start processing
INFO 2023/09/13 22:06:27 [mouse_monitor.py:228] save screenshot
INFO 2023/09/13 22:06:27 [mouse_monitor.py:261] screenshot coord：[0, 0, 0, 0]
WARNING 2023/09/13 22:06:27 [pool_exception_util.py:17] An error occurred, see error.log

Of course, the above It's just my speculation based on the documentation and source code. I hope you can reply. Thank you again!

How I get Win10 Scaling

def get_physical_resolution():
    hDC = win32gui.GetDC(0)
    wide = win32print.GetDeviceCaps(hDC, win32con.DESKTOPHORZRES)
    high = win32print.GetDeviceCaps(hDC, win32con.DESKTOPVERTRES)
    return {"wide": wide, "high": high}

def get_virtual_resolution():
    wide = win32api.GetSystemMetrics(0)
    high = win32api.GetSystemMetrics(1)
    return {"wide": wide, "high": high}

def get_win10_scaling_old():
    """
I know this method will conflict with SetProcessDpiAwareness(2), so I have changed the acquisition method. This is the method at that time.
    """
    warnings.warn('This method will conflict with mss setting dpi awareness and is no longer used. Only the writing method is retained.', DeprecationWarning)
    real_resolution = get_physical_resolution()
    screen_size = get_virtual_resolution()
    proportion = round(real_resolution['wide'] / screen_size['wide'], 2)
    return proportion

mouse_monitor.py simplified example

import mss
import yaml
import time
import logging
import mss.tools
import pynput.mouse as pm
import utils.active_window as aw
from utils.conf_util import load_conf
from utils.thread_pool_util import THREAD_POOL
from utils.screenutils import get_scaling, get_win_version
from utils.win10_scaling_util import initialize_dpi_awareness
from utils.pool_exception_util import ThreadPoolExceptionLogger

base_conf = {}
save_timestamp_list = []

def start_catch_mouse():
    if get_win_version() == 'win10':  # win7 try to get shcore will raise exception
        initialize_dpi_awareness()

    pml = pm.Listener(on_click=on_click)
    pml.start()

def loading_conf():
    global base_conf
    base_conf = load_conf()

def on_click(x, y, button, pressed):
    if not pressed and button.name == 'left':
        loading_conf()
        # listen program name
        listen_exe_name = base_conf['exeName'].strip().lower()
        pname, wtext = aw.get_active_window_process_name()
        if pname.strip().lower() == listen_exe_name:
            # sys scaling
            scaling = get_scaling()

            logging.info(f'sys scaling: {scaling}')
            # listen program title
            main_window_text = base_conf['mainWindowText'].strip().lower()
            if wtext.strip().lower() == main_window_text:
                rect = aw.get_active_window_client_rect()
                if rect and type(rect) == tuple:
                    logging.info(f'origin window rect：{rect}')
                    rect = list(rect)
                    rect[0] = rect[0] if rect[0] >= 0 else 0
                    rect[1] = rect[1] if rect[1] >= 0 else 0
                    rx = x - rect[0]
                    ry = y - rect[1]

                    if wtext.strip().lower() == main_window_text:  # 在主界面单击
                        dl_coord = base_conf['dataListCoord']
                        dl_minx = round(dl_coord[0] * scaling)
                        dl_miny = round(dl_coord[1] * scaling)
                        dl_maxx = round(dl_minx + (dl_coord[2]) * scaling)
                        dl_maxy = round(dl_miny + dl_coord[3] * scaling)
                        logging.info(
                            f'di area：min：({dl_minx},{dl_miny}) max：({dl_maxx},{dl_maxy}) click coord(relative)：({rx},{ry})')

                        if dl_minx <= rx <= dl_maxx and dl_miny <= ry <= dl_maxy:  # 在di区域点击
                            logging.info('clicked in di area, start processing')
                            # 新线程执行双击操作
                            THREAD_POOL.submit(ThreadPoolExceptionLogger(async_do_click), x, y, rect, scaling)

def async_do_click(x, y, rect):
    logging.info('save screenshot')
    save_img_id, adj_rect = save_fs_img(rect)
    save_data(x, y, save_img_id, adj_rect)
    global save_timestamp_list
    save_timestamp_list.append(save_img_id)

def save_fs_img(rect):
    save_img_dir = 'path/to/save/img'

    save_img_id = int(round(time.time() * 1000))
    logging.info(f'screenshot coord：{rect}')
    grab_coord = tuple(rect)
    img_path = f'{save_img_dir}/fs-{save_img_id}.png'
    with mss.mss() as m:
        img = m.grab(grab_coord)
        mss.tools.to_png(img.rgb, img.size, output=img_path)

    return save_img_id, grab_coord

def save_data(x, y, save_img_id, rect):
    data = {
        'x': x,
        'y': y,
        'rect': rect,
    }
    main_dir = base_conf['saveMainDir']
    data_dir = base_conf['saveDataDir']
    data_name = save_img_id
    with open(f'{main_dir}/{data_dir}/{data_name}.yml', 'w', encoding='utf8') as f:
        yaml.dump(data, f, allow_unicode=True)

How I get foreground window rect

def get_active_window_client_rect():
    hwnd = win32gui.GetForegroundWindow()
    client_rect = win32gui.GetClientRect(hwnd)
    left, top = win32gui.ClientToScreen(hwnd, (client_rect[0], client_rect[1]))
    right, bottom = win32gui.ClientToScreen(hwnd, (client_rect[2], client_rect[3]))
    return left, top, right, bottom

Upvote & Fund

Fund with Polar

tannergilligan commented 1 year ago

+1; this issue also happens for me after my program runs for ~3 hours.

CTPaHHuK-HEbA commented 1 year ago 
  From log:
  > INFO 2023/09/13 22:06:27 [mouse_monitor.py:120] origin window rect：(0, 0, 0, 0)

Source mss:

class MSS(MSSBase):
    def __init__(self, /, **kwargs: Any) -> None:
       ...
        self._handles.region_width_height = (0, 0)

        if self._handles.region_width_height != (width, height):
            ...
            self._handles.data = ctypes.create_string_buffer(width * height * 4)  # [2]

May be: if first cordinate (width, height) == (0, 0); self._handles.data - not set and Error.

Need: ?

    def __init__(self, /, **kwargs: Any) -> None:
       ...
        self._handles.region_width_height = (None, None)
BoboTiG commented 1 year ago

Before changing the current code, we will need a regression test so that we can validate both the issue and the fix.

Tratux commented 1 year ago

mss.exception.ScreenShotError: gdi32.GetDIBits() failed.

I get the same error after using 'mss' with 'threading.Timer' for a while. My code uses mss to get one pixel from the screen at the same place in a loop.

BoboTiG commented 1 year ago

@Tratux can you share a minimal reproduction code please?

Tratux commented 1 year ago

@BoboTiG I've tested and I got the same error with this code. The code executed for at least 10 min without an error.

import mss
import threading
import numpy as np

def loopFunc():
    pix = getPixel(100, 100)
    print(pix)

    timer = threading.Timer(0.1, loopFunc)
    timer.start()

def getPixel(x, y):
    region = (x, y, x+1, y+1)
    pixel = np.asarray(mss.mss().grab(region))
    return pixel

if __name__ == "__main__":
    loopFunc()
CTPaHHuK-HEbA commented 12 months ago

@Tratux This bad code: Intro Thread, create new Thread and new ...

If run in Python 3.12 After 2 cycle

  Exception in thread Thread-1:
  ...
  File "C:\Program Files\Python312\Lib\threading.py", line 971, in start
     _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown

If run in 3.11 After some time:

Exception in thread Thread-4998:
Traceback (most recent call last):
  File "D:\Program Files\Python\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "D:\Program Files\Python\Lib\threading.py", line 1394, in run
    self.function(*self.args, **self.kwargs)
  File "D:\Program Files\Python\t.py", line 6, in loopFunc
    pix = getPixel(100, 100)
          ^^^^^^^^^^^^^^^^^^
  File "D:\Program Files\Python\t.py", line 14, in getPixel
    pixel = np.asarray(mss.mss().grab(region))
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Program Files\Python\Lib\site-packages\mss\base.py", line 90, in grab
    screenshot = self._grab_impl(monitor)
                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Program Files\Python\Lib\site-packages\mss\windows.py", line 252, in _grab_impl
   raise ScreenShotError("gdi32.GetDIBits() failed.")
mss.exception.ScreenShotError: gdi32.GetDIBits() failed.

For fast: threading.Timer(0.001, loopFunc) Too: Exception in thread Thread-4998:

Im run Process Explorer: and saw after GDI handles ~ 10K python.exe crash.

This Windows limit GDI handles. https://learn.microsoft.com/uk-ua/windows/win32/sysinfo/user-objects?redirectedfrom=MSDN HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\USERProcessHandleQuota My Win11 limit - 10K

Also in my Windows Event Viewer: Windows Logs->Application Windows Error Reporting : Event: GDIObjectLeak python.exe https://stackoverflow.com/questions/8302287/how-to-debug-gdi-object-leaks

CTPaHHuK-HEbA commented 12 months ago

The Tratux error is not the same as the original error.

For first issue : If run a test from #270 for the current version

import mss

with mss.mss() as sct:
    region0 = {"top": 0, "left": 0, "width": 0, "height": 0}
    sct.grab(region0)

Same error:

  File "D:\Program Files\Python\Lib\site-packages\mss\windows.py", line 250, in _grab_impl
   bits = gdi.GetDIBits(memdc, self._handles.bmp, 0, height, self._handles.data, self._handles.bmi, DIB_RGB_COLORS)
                                                             ^^^^^^^^^^^^^^^^^^
AttributeError: '_thread._local' object has no attribute 'data'
dbwrush commented 8 months ago

I'm seeing the same error message here, mss.exception.ScreenShotError: gdi32.GetDiBits() failed. It was working fine for a few hours and then stopped. Here is the relevant code.

        def __init__(self):
            self.observation_space = gymnasium.spaces.Box(low=0, high=255, shape=(1, 83, 100), dtype=np.uint8)
            self.action_space = gymnasium.spaces.Discrete(3)
            self.cap = mss
            self.game_location = {'top': 300, 'left': 0, 'width': 600, 'height': 500}
            self.done_location = {'top': 405, 'left': 340, 'width': 660, 'height': 70}
        def get_observation(self):
            raw = np.array(self.cap.grab(self.game_location))[:, :, :3] #Error occurs here!
            gray = cv2.cvtColor(raw, cv2.COLOR_BGR2GRAY)
            resized = cv2.resize(gray, (100, 83))
            channel = np.reshape(resized, (1, 83, 100))
            return channel