search

justingardner / mgl

A suite of mex/m files for displaying psychophysics stimuli
http://justingardner.net/mgl
Other
18 stars 22 forks source link

Testing with Apple Silicon / Matlab R2023b / macOS Sonoma 14.2 #89

Closed benjamin-heasly closed 10 months ago

benjamin-heasly commented 10 months ago

This issue is a summary of MGL testing that I did on an Apple Silicon (M1) machine with the Silicon-native version of Matlab R2023b on macOS Sonoma 14.2. I'm hoping that putting all this info into an issue will make it easy to search for.

Short summary: I rebuilt and tested the MGL socket-related mex functions and other mex functions. I ran tests of rendering commands with onscreen and offscreen rendering and compared to known snapshots and/or general expectations. I also ran through the MGL Retinotopy task. I found several breaking issues but I was able to address them all. So to my knowledge, everything is working!

I added several comments below with specifics of what I tested and the changes I made. The changes are all included with PR #90. When we merge that PR, I think we should close this issue.

benjamin-heasly commented 10 months ago

System info

Here's info about the system I tested on.

Hardware

2021 iMac Built-in 24-inch display, no additional displays Apple M1 chip 8GB memory

Software

macOS Sonoma 14.2 (23C64) (latest as of 14 December 2023)

Mgl git commit e7b1fd17f86299d2735060c3fd018cb69478cfa4 (head of the default master branch, accessed 14 December 2023)

mrTools git commit eecfb1b (head of the default master branch, accessed 14 December 2023)

Matlab R2023b for Apple silicon

Java 8 (required for R2023b silicon) Amazon Corretto installation as recommended by Mathworks Amazon Corretto 8.392.08.1

>> ver
-----------------------------------------------------------------------------------------
MATLAB Version: 23.2.0.2428915 (R2023b) Update 4
MATLAB License Number: DEMO
Operating System: macOS  Version: 14.2 Build: 23C64 
Java Version: Java 1.8.0_392-b08 with Amazon.com Inc. OpenJDK 64-Bit Server VM mixed mode
-----------------------------------------------------------------------------------------
MATLAB                                                Version 23.2        (R2023b)
benjamin-heasly commented 10 months ago

Socket functions

Rebuilding the socket functions required minor changes to acknowledge the new mexmaca64 mex function extension and related naming: 4fd2f23628f998c4637fd35a9096d8354054b944

With that, socket function building and testing was smooth.

>> mglMakeSocket
...
MEX completed successfully.

>> mglTestSocket
Testing sockets...
...
...Sockets tests passed OK!

>> mglTestMultipleSockets
Testing 3 sockets at once
...
OK (0.000376 seconds)
benjamin-heasly commented 10 months ago

Other mex functions

Rebuilding other mex functions also required minor changes for mexmaca64. I also removed the QTKit framework from the build, since this produced warnings like ld: warning: ignoring file ... missing required architecture arm64 in file, and as far as I know is not required for MGL Metal: 63711ffbc1e868daecb2a23764803052531d3dd0.

With that, building was smooth.

>> mglMakeMetal(1)
...
= Compilation SUCCESSFUL for mglUnbindFrameBufferObject.c ==
benjamin-heasly commented 10 months ago

mglRunRenderingTests

I ran our automated rendering tests to compare render command results to known snapshots. This raised several errors which I addressed in a few different categories, below.

seems OK

mglTestFillOval and mglTestMetalDots had results that didn't compare exactly to saved snapshots. But by eye the results were indistinguishable, so I updated the snapshots: 4d6626601cdc7dcf861145caa8ae0dfb5c5926a8

coordinate flip

mglTestScreenCoordinates and mglTestTextureOddSizes were flipped upside-down but otherwise looked correct. I think the flip is expected given changes in 28792d49de26dc57c62dca69ee5f2f9b2a85cd33. So, I updated the snapshots to accept the flip: 4fd04b04a6613dde7eb7bb868fe23a2601d27927

arcs

mglTestGluAnnulus, mglTestGluPartialDisk, and mglTestMetalArcs had over-sized, blurry, and cut-off output. I think this was partially expected given changes to arcs functionality starting with 80bab2520c4266d73b01010944aca85bb75f300d. I was able to correct the size and blur by updating the tests to match new arcs command behavior.

The image cut-off seemed to be a bug where the arcs code assumed the render target had the same size as the onscreen view, even when the target was an offscreen texture. With changes in 7213c4e5406a8baab834ce53197ae34ad2e69120 and updated Metal app binary in 9efac17f1a70c5fc7f0b0637077d200ef4fb48b5, these tests look OK again.

text rendering

mglTestMetalText, mglTestTextDraw, and mglTestTextureMultiple crashed Matlab with a system dialog saying This version of "MATLAB" is not compatible with macOS 14 or later and needs to be updated.

Screenshot 2023-12-14 at 2 07 53 PM

I tracked the issue down to deprecated calls that mglText makes to Apple's ATSUI text rendering API. As of macOS 14 Sonoma this API is now expected to raise dialogs like this and to terminate the process (see release notes)!

I created #91 for migrating to the Core Text API instead.

I also created a pure Matlab workaround to use in the meantime: 6d64d7ca7b24fb37e869d3f1e6e6ab5f7ac63513

With all that, mglRunRenderingTests is passing and looking good to me.

>> mglRunRenderingTests

Running 32 tests.
...
All tests passed!
ans = 
  1×32 struct array with fields:
    snapshot
    renderedImage
    isSuccess
    testName
    snapshotData
benjamin-heasly commented 10 months ago

mglRenderingDemo

I ran through mglRenderingDemo, which tests the same drawing commands as mglRunRenderingTests above, but uses mglPause to show results interactively.

I had a few hiccups getting mglPause to work. Instead of responding to key presses, mglPause repeatedly warned:

!!!(mglGetKeyEvent) 64bit version not implemented
(mglPrivateListener) !!! **WARNING** To get keyboard events, you must allow Terminal to 'control your computer' by going to System Preferences/Privacy/Accessibility and adding Terminal to the list of apps that are allowed to control your computer. See http://gru.stanford.edu/doku.php/mgl/beta#keyboard_events

The usual fix to enable control your computer didn't work on its own.

I updated mglPrivateListener to use the supported AXIsProcessTrusted() Apple API call instead of the deprecated AXAPIEnabled() in d9aec9c2f014c67abdd78cfedb7b4417ef63488c. I suspect (not sure) that under Sonoma the deprecated call is no longer functional and always returns false.

Then in System Settings... -> Privacy and Security -> Accessibility I removed Matlab from the list of allowed applications, then re-added it. For some reason it didn't work just to enable Matlab, I had to remove it first.

Screenshot 2023-12-19 at 9 47 29 AM

With this mglPause no longer printed warnings, indicating Matlab had accessibility enabled OK. But it still didn't detect any key presses. It turns out Sonoma has an additional permission required, to enable Input Monitoring.

In System Settings... -> Privacy and Security -> Input Monitoring I added Matlab.

Screenshot 2023-12-19 at 10 09 03 AM

With all that, mglPause worked great.

benjamin-heasly commented 10 months ago

More rendering tests

With everything above, things were working pretty well. I'll note a few more successes:

benjamin-heasly commented 10 months ago

mglRetinotopy

Finally, I ran through the mglRetinotopy task. I looked at a few different stimuli using this Matlab script:

clear all

% Cool wedge stimulus, with calibration that shows text.
mglRetinotopy('wedges=1', 'numCycles=1', 'doEyeCalib=-1', 'dispText=Test');

% Some cool stimuli.
mglRetinotopy('rings=1', 'numCycles=1', 'doEyeCalib=0');
mglRetinotopy('bars=1', 'numCycles=1', 'doEyeCalib=0');
mglRetinotopy('barsTask=1', 'numCycles=1', 'doEyeCalib=0');

mglClose();

Overall this ran smoothly and responded to different key presses as prompted in the Matlab command window.

The last stimulus for barsTask looked unfamiliar and probably incorrect. I believe the interpretation of "dot size" has changed from pixels to degrees visual angle. With a corresponding change to the task's default dotSize in 17da4d0471f4888f75bd4ec0ef9f909f7bae73ec the task now looks good -- at least to my eye!