Supporting Python 3.10+

tbenst commented 1 year ago

Currently, psychopy only builds reliably on Python 3.8, as this is what is used by the test suite as written in Jun 2020. I've successfully built PsychoPy on Ubuntu up to Python 3.10, and on macOS up to Python 3.8.

Python 3.8 received its final bugfix in May, 2021.

As of Nov 2022, only Python 3.10 and 3.11 receive full support.

Psychopy is a fantastic package and would be great to ensure the next generation of users can install the package by adopting the best supported Python versions, namely 3.10 and 3.11.

mdcutone commented 1 year ago

We're having a hard time supporting Python 3.10+ on Ubuntu because implicit casting of numeric type arguments has being removed. This breaks wxPython and there hasn't been a fix released yet.

tbenst commented 1 year ago

wxPython from conda-forge has a release for Python 3.10 and 3.11: https://anaconda.org/conda-forge/wxpython. This branch of PsychoPy builds on Ubuntu 22.04 when using wxPython from conda forge: https://github.com/tbenst/psychopy/tree/2022.2.4_tyler. The only change is removing two dependencies.

peircej commented 1 year ago

Hi @tbenst and thanks for you work on this. We're keen to see support for 3.10 so if you're able to help get this working that's great. We don't think there's anything about PsychoPy's own codebase that prevents 3.10 usage. As @mdcutone notes, one issue is that we've seen problems with wxPython on 3.10 and the other is that hardware manufacturers (e.g. eyetrackers) haven't all built their dylibs for 3.10 yet but these are the only things holding us back.

Regarding conda, it would be great to see a condaforge deployment that actually works. My understanding before was that psychtoolbox didn't compile there (how did you get around that? or does it just work now?) and we've also seen a number of incompatible conda packages, historically. It would be great to see it work but our team haven't had time to debug the issues.

For your changes

removing pyo won't be needed in future as that is removed from psychopy itself from 2023.1 onwards.
removing javascripthon is an issue because that's required for transpiling Code Components into JS for online studies. Then again Javascripthon is now tested on Py3.10 since they removed the broken macropy dependency. It looks like this might be solved by getting this pull request pulled into conda-forge feedstock

LMBooth commented 1 year ago

Is there an expected release date for 2023.1 or is it until this issue is fixed?

danielshub commented 1 year ago

I got some time to play around trying to get 2023.1.1 to work on Arch Linux with Python 3.10. I basically ran python setup.py build and python setup.py install --optimize=1 --skip-build, installed the packages in setup.cfg, created the psychopy user group and set the ulimits. The only real gotchas are:

I did not install ffpyplayer or egi-pynetstation since they are not available in Arch
I did not install psychopy-legcay-mic and psychopy-visionscience so I have no idea if those work with Python 3.10.
I needed to install pyglet v1.5.27 instead of v2.0.5 (which is documented but will potentially cause problems in the future)
I needed to install an unreleased version of moviepy (head of the master branch from github) since v1.03 has some outdated dependencies
I needed to install xmlschema even though it is not listed in setup.cfg.
I needed to install webkit2gtk, flac, and which on a very bare bones Arch Linux install. I am not sure why I needed the which package and it may not strictly be needed.
I needed to edit app/builder/dialogs/__init__.py and app/builder/builder.py to make them compatible with wxpython/Python 3.10.

In __init__.py I changed all occurrences of wx.Size(self.valueWidth to wx.Size(int(self.valueWidth) while in builder.py there were a dozen or so additions of int related to self.iconXpos, xSt, w, and h. I can submit a PR, but I don't know if you would prefer to take a minimal change approach and do the typecast only at the line where the unit tests fail or a more global approach of trying to figure out where the values are being set and do the type cast there. It seems like the current approach that has been taken is just to randomly spam int().

Once built and installed I ran the unit tests. There are a bunch of errors and failures. Some of the errors and failures seem to be related to me missing some data files. I am not sure where I download those from. Until I get those files, I am not sure if those parts work with Python 3.10. There are a few failures that I think might potentially be related to Python 3.10, but they could be my system or user error

FAILED test_experiment/needs_wx/test_Experiment.py::TestExpt::test_Exp_LoadCompilePsyexp - AttributeError: 'TestExpt' object has no attribute '_checkCompile'
FAILED test_hardware/test_emulator.py::TestLaunchScan::test_launch_scan - NameError: name 'launchScan' is not defined
FAILED test_hardware/test_emulator.py::TestLaunchScan::test_no_mode - NameError: name 'ResponseEmulator' is not defined
FAILED test_hardware/test_emulator.py::TestLaunchScan::test_sync_generator - NameError: name 'SyncGenerator' is not defined
FAILED test_iohub/test_computer.py::TestComputer::test_getTime - AttributeError: type object 'Computer' has no attribute 'currentSec'
FAILED test_iohub/test_computer.py::TestComputer::test_procPriority - AttributeError: type object 'Computer' has no attribute 'getProcessPriority'. Did you m...
FAILED test_iohub/test_event_get_and_clear.py::testGlobalBufferOnlyClear - AttributeError: 'NoneType' object has no attribute 'lower'
FAILED test_iohub/test_event_get_and_clear.py::testDeviceBufferOnlyClear - assert 29 == 1
FAILED test_iohub/test_event_get_and_clear.py::testAllBuffersClear - RuntimeError: An existing ioHubConnection is already open. Use ioHubConnection.getActiv...
FAILED test_visual/test_glfw_backend.py::test_open_glfw_window - ValueError: Only one kind of `winType` can be used per session.

I also get a handful of: AssertionError: Your frame period is 33.3ms which suggests you aren't syncing to the frame which seems correct since my refresh rate is set to 30 Hz so maybe the test is making an assumption?

peircej commented 1 year ago

Thanks for documenting this Dan

Regarding movies:

pyglet 2.0.5 is indeed one that we need to update for
we really want ffpyplayer to become the single supported engine, so I hope there's a way to get that working on Arch. moviepy does not have as high performance and we'll be dropping that as a dependency if possible
xmlschema is only needed for testing which is why it isn't in setup.cfg from 2023.2 (i.e. in the dev branch now) we're moving to pyproject.toml using pdm and that allows us to distinguish between these

Good to know that you got wxpython working with 3.10. I think calling int() periodically is fine. I guess ideally we'd be checking that floats are never created with integer divisions but this isn't timing-critical stuff while creating dialogs so no big deal. If you've got the code already set for that then sending a PR would be great

On the failures:

test_emulator failing is fine - we've moved the emulator out to a plugin but not moved the tests
Similarly for glfw window backend - now a plugin
iohub tests don't normally get run on the CI runs (because none of the hardware is available) which probably means the tests are out of sync with the code
the -checkCompile issue looks like confusion caused by duplicated test code

mdcutone commented 1 year ago

We're working on this right now. We got PsychoPy running on Python 3.10 and wxPython 4.2 in a branch off dev. Still working through the OpenGL stuff to support recent pyglet releases.

cbrnr commented 1 year ago

Quick update, I'm also running Arch Linux, and the current dev branch seems to work fine even with Python 3.11. If anyone wants to try, here are the required steps (I'm using a virtual environment, because it does not require root privileges and keeps the system env clean):

Install the following two packages: sudo pacman -S python-wxpython webkit2gtk
Change to the desired directory (e.g. cd ~/venvs) and create a venv: python -m venv --system-site-packages psychopy
Activate the venv: source psychopy/bin/activate
Install PsychoPy (here I'm installing the dev branch, but once a new release is available, this simplifies to pip install psychopy): pip install git+https://github.com/psychopy/psychopy --ignore-requires-python
Start PsychoPy: psychopy

Given that this seems to work with Python 3.11, it would be great if you considered bumping requires-python to <3.12 (then --ignore-requires-python could be dropped from the command in step 4).

psychopy / psychopy

Supporting Python 3.10+ #5216