Slicer / SlicerLanguagePacks

3D Slicer extension for creating, editing, and storing translations for Slicer core and extensions
MIT License
8 stars 11 forks source link

Add screenshot creator function that captures widget metadata #25

Open lassoan opened 1 year ago

lassoan commented 1 year ago

Tutorials require drawing arrows, rectangles, and text on screenshots. For example the tutorial slide may require "drawing of an arrow with label 1, pointing to the center of OK button".

Position of these annotations needs to be specified relative to widget positions because the layout and widget sizes depend on screen size, font size, chosen language, etc. Therefore, when we capture screenshots for a tutorial then we also need to record name, position and size (the visible rectangle) of all visible widgets. This information should be stored in json in a separate file, but preferably inside the screenshot file, in a comment field.

To identify a widget, we could rely on Qt object names. Each Qt widget has an object name, which is a simple string that is specified when the widget is created and it does not change when the language of the application is changed. The name of a widget is not guaranteed to be unique in the whole application GUI, but it can be used to identify a widget among children of a widget. Therefore the widget could be identified with a "path" that contains names of parent widgets. For example: qSlicerMainWindow/CentralWidget/CentralWidgetLayoutFrame/qMRMLSliceWidgetYellow/SliceController/BarWidget/SliceOffsetSlider

Task: Implement a Python function that captures a screenshot and embeds widget metadata in the image file. Add it to the current LanguageTools module logic. No need for GUI for now.

lassoan commented 1 year ago

Code snippet that captures information about all visible widgets:

def widgetPath(widget):
    path = ""
    while widget:
        path = (widget.objectName if widget.objectName else "?") + ("/" + path if path else "")
        widget = widget.parent()
    return path

mainWindow = slicer.util.mainWindow()
mainWindowPos_global = mainWindow.mapToGlobal(mainWindow.rect.topLeft())

widgetsInfo = []
widgets = slicer.util.findChildren()
for widget in widgets:
    if hasattr(widget, "isVisible") and widget.isVisible() and hasattr(widget, "mapToGlobal"):
        widgetTopLeft_global = widget.mapToGlobal(widget.rect.topLeft())
        widgetBottomRight_global = widget.mapToGlobal(widget.rect.bottomRight())
        widgetPos_mainWindow = [widgetTopLeft_global.x() - mainWindowPos_global.x(), widgetTopLeft_global.y() - mainWindowPos_global.y()]
        widgetSize_mainWindow = [widgetBottomRight_global.x() - widgetTopLeft_global.x(), widgetBottomRight_global.y() - widgetTopLeft_global.y()]
        widgetInfo = {
            "widget": widgetPath(widget),
            "className": widget.className(),
            "position": widgetPos_mainWindow,
            "size": widgetSize_mainWindow,
            }
        if hasattr(widget, "windowTitle") and widget.windowTitle:
            widgetInfo["windowWitle"] = widget.windowTitle
        if hasattr(widget, "text") and widget.text:
            widgetInfo["text"] = widget.text
        widgetsInfo.append(widgetInfo)

Resulting widgetsInfo:

[
    {"widget": "qSlicerMainWindow", "position": [0, 0], "size": [1919, 1032]},
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar",
        "position": [0, 97],
        "size": [1919, 49],
    },
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar/?",
        "position": [765, 111],
        "size": [349, 21],
    },
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar/?/?",
        "position": [765, 111],
        "size": [349, 21],
    },
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar/qMRMLSequenceBrowserSeekWidget",
        "position": [414, 108],
        "size": [349, 26],
    },
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar/qMRMLSequenceBrowserSeekWidget/label_IndexUnit",
        "position": [758, 114],
        "size": [-1, 14],
    },
    {
        "widget": "qSlicerMainWindow/SequenceBrowserToolBar/qMRMLSequenceBrowserSeekWidget/label_IndexValue",
        "position": [755, 114],
        "size": [-1, 14],
    },
... a total of 183 widgets
]