Media player view size and design size issues

[rh101]

• axmol version:
• devices test on: iPhone/Windows
• developing environments
  • NDK version: r23c
  • Xcode version: 14.2+
  • Visual Studio:
    • VS version: 2022 (17.9+)
    • MSVC version: 19.39+
    • Windows SDK version: 10.0.22621.0+
  • cmake version: Steps to Reproduce:

Add the 1280x720.mp4 video from cpp-tests to a fresh project.

Add this code to MainScene.cpp init() method:

auto* videoPlayer = ui::VideoPlayer::create();
videoPlayer->setPosition(visibleSize / 2);
videoPlayer->setAnchorPoint(Vec2::ANCHOR_MIDDLE);
videoPlayer->setContentSize(visibleSize / 2);
videoPlayer->setFileName("1280x720.mp4");
videoPlayer->setLooping(true);
videoPlayer->play();
videoPlayer->setKeepAspectRatioEnabled(true);
videoPlayer->setUserInputEnabled(true);
addChild(videoPlayer);

In AppDelegate.cpp, replace the applicationDidFinishLaunching method with the code here:

bool AppDelegate::applicationDidFinishLaunching()
{
    // initialize director
    auto director = Director::getInstance();
    auto glView   = director->getGLView();
    if (!glView)
    {
#if (AX_TARGET_PLATFORM == AX_PLATFORM_WIN32) || (AX_TARGET_PLATFORM == AX_PLATFORM_MAC) || \
    (AX_TARGET_PLATFORM == AX_PLATFORM_LINUX)
        glView = GLViewImpl::createWithRect("rendertest", ax::Rect(0, 0, 1080, 1920));
#else
        glView = GLViewImpl::create("rendertest");
#endif
        director->setGLView(glView);
    }

    // turn on display FPS
    director->setStatsDisplay(true);

    // set FPS. the default value is 1.0/60 if you don't call this
    director->setAnimationInterval(1.0f / 60);

    // Set the design resolution
    glView->setDesignResolutionSize(720, 1280, ResolutionPolicy::SHOW_ALL);

    // create a scene. it's an autorelease object
    auto scene = utils::createInstance<MainScene>();

    // run
    director->runWithScene(scene);

    return true;
}

Now, with videoPlayer->setKeepAspectRatioEnabled(true);, this happens when you run the app:

No touch input works, since it is all out of alignment, and touching other areas of the screen triggers the media controls.

If I minimize the window, and bring it back up, it changes to this:

All touch works correctly, since the control images are exactly aligned with the touch locations. Full-screen switching also works.

With videoPlayer->setKeepAspectRatioEnabled(false);:

Same problem with touch input, controls cannot be clicked on, since the actual touch positions are not aligned with the controls.

If you minimize the window, and bring it back up, this happens:

All controls and touch positions are aligned now, and full-screen mode works as expected, for example:

So, what happens when the window is minimized and then brought back to the foreground. Whatever is happening there is fixing the problem with the video player.

Now, the issue of the controls being out of the video player window when videoPlayer->setKeepAspectRatioEnabled(true) is in PR #1851, so it now looks like this (after window is minimized then brought back to foreground):

[rh101]

@halx99 Sorry, I think I confused things a bit. PR #1851 does not address the problem described in this issue, it's just for adjusting the visual position of the media controls to cater for the aspect ratio of the video.

The issue described here isn't related to the media controls, but rather to something being incorrect with the glview when view size is greater than design size, which is also affecting the touch input locations (for example, view size 1080,1920, and design resolution 720,1280). It only fixes itself if the window is minimized (put into background) then brought back (moved to foreground).

Perhaps it's not even a media player issue, but something more general.

[rh101]

A quick look into this showed the following, and this is when using a desktop 2560x1440 resolution.

When the app starts up, this method is called:

void GLViewImpl::onGLFWWindowSizeCallback(GLFWwindow* /*window*/, int w, int h)
{
    if (w && h && _resolutionPolicy != ResolutionPolicy::UNKNOWN)
    {
        handleWindowSize(w, h);

#if defined(AX_USE_METAL)
        // update metal attachment texture size.
        int fbWidth, fbHeight;
        glfwGetFramebufferSize(_mainWindow, &fbWidth, &fbHeight);
        backend::UtilsMTL::resizeDefaultAttachmentTexture(fbWidth, fbHeight);
#endif

        Size size(w, h);
        Director::getInstance()->getEventDispatcher()->dispatchCustomEvent(GLViewImpl::EVENT_WINDOW_RESIZED, &size);
    }
}

Which eventually leads to this:

void GLView::updateDesignResolutionSize()
{

// HERE _screenSize.width= 1080, and _screenSize.height = 1920

    if (_screenSize.width > 0 && _screenSize.height > 0 && _designResolutionSize.width > 0 &&
        _designResolutionSize.height > 0)
    {
        _scaleX = (float)_screenSize.width / _designResolutionSize.width;
        _scaleY = (float)_screenSize.height / _designResolutionSize.height;

        if (_resolutionPolicy == ResolutionPolicy::NO_BORDER)
        {
            _scaleX = _scaleY = MAX(_scaleX, _scaleY);
        }

        else if (_resolutionPolicy == ResolutionPolicy::SHOW_ALL)
        {
            _scaleX = _scaleY = MIN(_scaleX, _scaleY);   // _scaleX and _scaleY are both 1.5, so this ends up as _scaleX = 1.5, which is not correct
        }
...

The vertical height of 1920 is actually not correct, and is resulting in a _scaleY of 1.5, which is also not correct. No touch input works because it is all out of alignment etc..

When the window is minimized, and then maximized, the following happens in GLView::updateDesignResolutionSize(), the window size somehow fixes itself, and it is now 1080, 1421:

So this:

_scaleX = (float)_screenSize.width / _designResolutionSize.width;  //= 1080 / 720 = 1.5
_scaleY = (float)_screenSize.height / _designResolutionSize.height; // = 1421 / 1280 = 1.11015630

So this:

else if (_resolutionPolicy == ResolutionPolicy::SHOW_ALL)
{
    _scaleX = _scaleY = MIN(_scaleX, _scaleY);   // _scaleX = 1.11015630
}

The final scale of 1.11015630 is correct, and the entire screen is now aligned correctly, and all touch input now works as expected.

[rh101]

@halx99 Possible fix by adding the following code to bool GLViewImpl::initWithRect(std::string_view viewName, const ax::Rect& rect, float frameZoomFactor, bool resizable) after the call to glfwCreateWindow:

int actualWidth, actualHeight;
glfwGetWindowSize(_mainWindow, &actualWidth, &actualHeight);
if (static_cast<int>(windowSize.width) != actualWidth || static_cast<int>(windowSize.height) != actualHeight)
{
    windowSize.set(static_cast<float>(actualWidth), static_cast<float>(actualHeight));
}

In the example above, attempting to set 1080,1920 actually resulted in a real window size of 1080,1421, so overwriting windowSize with the true size returned by glfwGetWindowSize fixed the problem. I have only tested this on Windows so far.

[halx99]

ok, could create pr?

获取Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: RH @.> Sent: Friday, April 26, 2024 11:30:13 PM To: axmolengine/axmol @.> Cc: Deal @.>; Mention @.> Subject: Re: [axmolengine/axmol] Media player view size and design size issues (Issue #1850)

@halx99https://github.com/halx99 Possible fix (only tested on Windows so far), adding the following code to bool GLViewImpl::initWithRect(std::string_view viewName, const ax::Rect& rect, float frameZoomFactor, bool resizable) after the call to glfwCreateWindow:

int actualWidth, actualHeight; glfwGetWindowSize(_mainWindow, &actualWidth, &actualHeight); if (static_cast(windowSize.width) != actualWidth || static_cast(windowSize.height) != actualHeight) { windowSize.set(static_cast(actualWidth), static_cast(actualHeight)); }

image.png (view on web)https://github.com/axmolengine/axmol/assets/8603230/d4b49c6c-3edb-45e7-9f9b-d5b66597f4b9

In the example above, attempting to set 1080,1920 actually resulted in a real window size of 1080,1421, so overwriting windowSize with the true size returned by glfwGetWindowSize fixed the problem. I have only tested this on Windows so far.

― Reply to this email directly, view it on GitHubhttps://github.com/axmolengine/axmol/issues/1850#issuecomment-2079619702, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABVHOJ5O4GAJQ6ICLCKORL3Y7JXILAVCNFSM6AAAAABGZE7BB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGYYTSNZQGI. You are receiving this because you were mentioned.Message ID: @.***>

[rh101]

This issue seems to be fixed, so unless anyone has any objections, I'm closing it.