search

LABSN / expyfun

Experimental paradigm functions.
BSD 3-Clause "New" or "Revised" License
13 stars 21 forks source link

CI: Test video explicitly #437

Closed larsoner closed 2 years ago

larsoner commented 2 years ago

@drammock looking at Azure logs like this I don't see any skips because of missing ffmpeg, which suggests that things work there:

https://dev.azure.com/LABSN/expyfun/_build/results?buildId=328&view=logs&jobId=db093384-e8eb-5bcc-f0a9-2a8f5e4220af&j=db093384-e8eb-5bcc-f0a9-2a8f5e4220af&t=b83569e5-86aa-5f40-c5f2-cb2c66a2c238

This PR explicitly adds a test that the video example runs. On my Linux machine, it pops up, the video plays, and then it exits. The same should in principle happen on Windows. Let's see...

If it does, @drammock feel free to play with your local setup / deps until it works, too, and push to this PR to update any instructions.

Closes #436

larsoner commented 2 years ago

Looks like the videos run:

2022-01-03 20:38:10,678 - INFO    - Expyfun: Initialization complete
2022-01-03 20:38:10,678 - EXP     - Expyfun: Participant: foo
2022-01-03 20:38:10,678 - EXP     - Expyfun: Session: foo
2022-01-03 20:38:18,907 - INFO    - Expyfun: Exiting
2022-01-03 20:38:18,965 - EXP     - Expyfun: Audio stopped and reset.

No idea why 3.9 fails (after the video test, during pytest), maybe it's spurious. I'll restart it...

drammock commented 2 years ago

I also have no problems playing the video on Linux. I opened #436 after an emailed support request, which I was able to reproduce in a windows VM. I'd wait for the interested party to chime in before spending any more time on this.

larsoner commented 2 years ago

You can also try python -c "import expyfun; expyfun._utils._has_video(raise_error=True)", it's what our CIs currently also do to check video support. It might tell you/them/us more information. The Pyglet traceback is not so useful IIRC because it just traverses a list of possible decoders, the last of which is WAV/RIFF, burying the useful FFMPEG error traceback.

JustinTFleming commented 2 years ago

Hey all, I messaged Dan about this a couple weeks ago. Struggling to track down what's going on because 1) recent pyglet commits over the past few months keep changing the behavior, and 2) I can no longer replicate my own RIFF ID error, possibly due to changing versions of pyglet and/or FFMPEG.

I ran Eric's suggested test with _has_utils() using three different pyglet versions since Novemeber, and they all produced the same output:

C:\Users\jtjus\AppData\Roaming\Python\Python37\site-packages\pyglet\media\codecs\wmf.py:838: 
  UserWarning: [WinError -2147417850] Cannot change thread mode after it is set
  warnings.warn(str(err))
Found FFmpegSource for new Pyglet

However, the behavior of simple_movie_demo.py is different depending on which of these pyglet versions I used. Note that I've altered the script to call an .mp4 file that I know has audio, but the behavior is similar with the example_video.mp4 included with expyfun.

While installing different versions of pyglet with pip install --user --upgrade, it's also possible that the version of FFMPEG I had installed was overwritten with whatever version is packaged with pyglet.

Apologies for the long comment. Happy to try any additional tests or potential fixes that may be helpful.

larsoner commented 2 years ago

1) recent pyglet commits over the past few months keep changing the behavior... 1fb5803 ... d1876a9

These are both commits to master. If you want to try newer Pyglet stuff between point releases, you need to use the 1.5-maintenance branch. master uses newer OpenGL than 1.5-maintenance, and is incompatible with how expyfun works (currently at least!).

I'm surprised that 69735d0 worked at all since it's also to master...

I would first try using 1.5.21 or earlier releases from pip (e.g., pip uninstall pyglet && pip install --no-cache pyglet==1.5.21), and only after that start looking into trying 1.5-maintenance commits (and never master commits).

larsoner commented 2 years ago

Okay so in my Windows VM, simple_video.py works properly on Pyglet 1.3.3. Maybe for now, use that?

We should keep this issue open, though, and figure out how to fix things on Windows. They work okay on Linux for me at least, so this might tell us about what we need to do to fix things on Windows...

JustinTFleming commented 2 years ago

Happy to work with an older version for now. After downgrading to pyglet 1.3.3, I am now getting a FFMPEG/AVbin not installed error. I downloaded and extracted a shared FFMPEG build, then copied and merged contents of the "bin", "lib", and "include" folders into the directory where python.exe lives (as per earlier directions from Dan) -- still get the error.

There is a note about possibly needing to copy DLL files from the System32 folder to the SysWOW64 folder. Some .dll files were included in the ffmpeg download (e.g. avcodec-58.dll, avformat-58.dll, etc.), but these don't show up in either the System32 or SysWOW64 folders. Any advice?

drammock commented 2 years ago

I downloaded and extracted a shared FFMPEG build, then copied and merged contents of the "bin", "lib", and "include" folders into the directory where python.exe lives (as per earlier directions from Dan)

I feel compelled to quote my own email at this point:

note that I'm calling the environment "test". We're about to do some dangerous things, be sure to do this in a new environment so that you don't mess up an environment that is already mostly working / being used for other experiments.

copying over the bin, lib, and include directories from the "full shared" build of ffmpeg was not meant to be a long-term solution. Quoting myself again:

What all this does is replace the version of ffmpeg that was installed automatically with pyglet, with a different version of ffmpeg that hopefully can properly open MP4 files. If it works, it will be a fragile fix, because if anyone runs "pip install --upgrade pyglet" in that environment, there's a chance that our special version of FFMPEG will get overwritten by pyglet's non-working version. So, please do open an issue on GitHub so that other devs who have real windows machines can help out in finding a more permanent solution.

As @larsoner pointed out here https://github.com/LABSN/expyfun/issues/436#issuecomment-1004334016 there is a package pyglet-ffmpeg that I wasn't aware of, which is meant to take care of this for you.

My advice would be to start over with a new conda environment (I can't stress that enough):

conda create -n test mne
conda activate test
pip install pyglet=1.3.3 pyglet-ffmpeg pyparallel rtmixer
pip install git+https://github.com/labsn/expyfun

and then try to run an unedited version of the expyfun video example. I know you want/need video-with-audio working, but debugging happens one step at a time. If we can get the example working with the default video file, then we can move on from there to address any issues you're still having with mp4 audio streams.

JustinTFleming commented 2 years ago

Ok, I deleted my previous conda environment and started again with a new one, as well as reverted to the original mp4 file in simple_movie_demo.py. pyglet-ffmpeg cannot be installed with pyglet 1.3.3, as it depends on pyglet>=1.4.0a1. I tried the above with pyglet 1.4.0a1, which failed again at the start of the playback loop (I can post the traceback if it's relevant). It seems like I need a way to get FFMPEG working with pyglet 1.3.3.

drammock commented 2 years ago

pyglet-ffmpeg cannot be installed with pyglet 1.3.3, as it depends on pyglet>=1.4.0a1

I was afraid of that. I'll try again tomorrow; i vaguely recall the system32 / SysWoW64 stuff from several years ago, probably if I step through it myself in my VM I'll remember how to get it to work

larsoner commented 2 years ago

I pushed some changes so that our code uses OpenGL directly rather than texture.blit from Pyglet. On Linux 1.5 at least it works for me. I then removed the enable_video keyword argument and related conditionals, as they are no longer necessary as they were workarounds just to work with blit I think.

It seems like I need a way to get FFMPEG working with pyglet 1.3.3.

pyglet < 1.4 uses avbin, so you'll need to get those binaries:

https://avbin.github.io/AVbin/Download.html

First I would try putting the 64-bit .dll in the examples/stimuli directory, cd to that directory, then python simple_video.py it and hope it works. If it does, then moving the avbin DLL in the System32 directory should also work. (The naming is confusing, but that's where 64-bit DLLs go.) This assumes you're running 64-bit Python on 64-bit Windows.

drammock commented 2 years ago

First I would try putting the 64-bit .dll in the examples/stimuli directory

the avbin installer is an EXE, and it doesn't give you a choice about where to unpack it; it goes straight to C:\Windows\System32\avbin64.dll. But putting it there worked (it no longer crashes with FileNotFoundError: Could not find module 'avbin64' (or one of its dependencies). Try using the full path with constructor syntax. However, it still crashes with pyglet.gl.lib.MissingFunctionException: glBindBuffer is not exported by the available OpenGL driver. VERSION_1_5 is required for this functionality. So I switched to the larsoner/video branch (this PR, at c9c216f) and tried again, now I get pyglet.gl.lib.MissingFunctionException: glCreateProgram is not exported by the available OpenGL driver. VERSION_2_0 is required for this functionality.

(note: this is in an environment that uses pyglet==1.3.3)

drammock commented 2 years ago

pyglet.gl.lib.MissingFunctionException: glCreateProgram is not exported by the available OpenGL driver. VERSION_2_0 is required for this functionality.

This may be a deal-breaker: according to the virtualbox forums:

The only way to get OpenGL 2.1 in a Windows guest is to install basic 3D (not WDDM), in safe mode, in an XP, Vista or Win7 guest. There is no other way to get accelerated OpenGL of any description.

@larsoner you said simple_video.py works in your windows VM with pyglet 1.3.3... is your VM running Windows 7 by any chance? (mine is Windows10)

Of course @JustiinTFleming is not in a VM so that ought not be a problem for that system... But if it won't work in a Windows 10 VM then I'm no longer of any use for testing :(

larsoner commented 2 years ago

now I get pyglet.gl.lib.MissingFunctionException: glCreateProgram is not exported by the available OpenGL driver. VERSION_2_0 is required for this functionality.

I think OpenGL 2.0 is a minimum for us -- anything other than video already uses glCreateProgram (text I think? but definitely shapes) should also have this problem. Can you confirm that experiments/drawing_methods.py also fails on your system (on main or here, doesn't matter since that example doesn't use video)?

I can confirm that video works on this branch on Pyglet 1.5.21 on macOS, too. Tomorrow I'll try on a native Windows machine with NVIDIA rather than my VM to make sure it's really a problem with Pyglet+Windows, and not Windows+MESA.

larsoner commented 2 years ago

@larsoner you said simple_video.py works in your windows VM with pyglet 1.3.3... is your VM running Windows 7 by any chance? (mine is Windows10)

Yeah my VM is Windows 7.

The only way to get OpenGL 2.1 in a Windows guest is to install basic 3D (not WDDM), in safe mode, in an XP, Vista or Win7 guest. There is no other way to get accelerated OpenGL of any description.

We don't actually need accelerated OpenGL, it can be software/unaccelerated. IIRC I got Mesa software rendering in Windows 10 VM at some point and it rendered okay. I can verify this and write some instructions... but I think it involves downloading some Mesa build compiled for Windows and putting the OpenGL DLL(s) in System32 (as usual), overwriting the system one(s).

In any case, I don't think we should let the VirtualBox Windows Guest OpenGL limitation stop us from using OpenGL 2.0. We already have been using it for most things other than video for years I think

drammock commented 2 years ago

Can you confirm that experiments/drawing_methods.py also fails on your system

confirmed.

In any case, I don't think we should let the VirtualBox Windows Guest OpenGL limitation stop us from using OpenGL 2.0.

agreed.

larsoner commented 2 years ago

@drammock if you want to mess with your Windows 10 system, try

https://fdossena.com/?p=mesa/index.frag

larsoner commented 2 years ago

On my native Windows system (with a good NVIDIA graphics card + drivers) video still doesn't work on 1.4+, but it does on 1.3. I did have to install the 64-bit Windows variant of https://avbin.github.io/AVbin/Download.html to get it to work. For now @JustinTFleming I suggest you do that. We can merge this PR as is I think, as it helps decouple us a bit more from Pyglet. I'll look into why 1.4/1.5 does not work on Windows.

Another option we might want to consider is loading video with imageio-ffmpeg, creating our own texture, and drawing that to the screen. In theory this might not be so hard, and actually should help us decouple (further) from Pyglet, which I think is a good thing for future maintainability.

drammock commented 2 years ago

We can merge this PR as is I think

in it goes, then!