search

VU-Cog-Sci / exptools2

Version 2 of exptools (by KnapenLab)
11 stars 9 forks source link

Proper releases #15

Open Gilles86 opened 5 years ago

Gilles86 commented 5 years ago

Hey guys,

Great work.

I'm starting a new project and developing an experiment from scratch . Since a year or two I try to do everything properly with with fixed Docker images and conda environments, but I keep spending a good portion of my precious postdoc time on duck taping backward compatibility issues of different Python packages. I think it would be super helpful if we could make different releases of exptools2 (rather than, e.g., commits) that we can use, so it's super clear with what exptools-code a given experiment was put together.

Github has a nice lemma on this: https://help.github.com/en/articles/creating-releases

Cheers, Gilles

lukassnoek commented 5 years ago

Yes, I agree. I held off any proper release because it's really still a pre-release. I definitely want @tknapen to look at the Eyetracker code (especially the default eyelink settings) before issuing a proper release. There are also some things to discuss w.r.t. the functionality of the package (i.e., are there still things missing?). Let's have a meeting about this when you guys are back from VSS?

Gilles86 commented 5 years ago

We both got back this morning, but I feel like a wet paper or something right now. Are you at the Spinoza next week?

I had a quick look at the new code yesterday at the Detroit airport, when rapidly prototyping an experiment. I really liked it. The only problem I had was that I still need 1-1.5 hours to get a working conda environment with exptools2 running. Maybe I am just an idiot, but an idiot with quite some training. One problem that I overlooked was that the (Sebastiaan Mathot's) -c cogsci psychopy-installation package automatically downgrades your Python installation to 2.x, which breaks exptools2. Maybe we can make a 'recommended' conda-environment for this package? You can upload those to your own anaconda-channel also. Would be neat.

The yaml settings-file is nicely streamlined now, in part by breaking backwards compatibility. The only thing I'm worried about is a use case that I always have. Namely, I need different settings for my laptop and for the stimulus computer. In the old yaml branch, I implemented an 'overloading'-environment trick to deal with this. This would look something like this:

global:
  monitor:
    size: [1024, 768]
  expt:
    n_trials: 100
scanner:
  monitor:
    size: [1920, 1080]

Any exptools experiment would then always ask which context you are in, before starting a session (where just pressing enter means the global environment). Any settings that you do not define for a specific environment (like scanner), are pulled out of the global-environment.

So whether you want to do this is a big design decision and I can imagine that maybe you and @tknapen think this is convoluted. However, I do know that this an issue that always comes up and the "no, wait I'll just quickly edit the settings-file for now, next time we'll make it a bit more structured. You are writing this down, right? Wait, I'll open a google doc. Is the scanner running?"-approach is flawed. The separate-branches-for-separate-stimulus-computers is slightly better but very clunky.

Cheers, Gilles

marcoaqil commented 5 years ago

Hey Gilles, just quickly, maybe i can help with the conda environment stuff. I did not encounter big problems installing and using exptools2 with python 3 both on my macbook and on the linux desktop at the 7T (there is an envionrment called Marco_prf or something like that which i have used to run my experiment which you can copy it if needed). I installed exptools2 normally with setup.py -develop, and then psychopy following the conda-relevant instructions on their website.

On the latter part, i also use different settings on macbook and linux. I also thought about different branches first. In the end my simple solution was having different setting files in my experiment repository. and then pick the relevant one when i run the experiment from the command line.

Gilles86 commented 5 years ago

Hey Gilles, just quickly, maybe i can help with the conda environment stuff. I did not encounter big problems installing and using exptools2 with python 3 both on my macbook and on the linux desktop at the 7T (there is an envionrment called Marco_prf or something like that which i have used to run my experiment which you can copy it if needed). I installed exptools2 normally with setup.py -develop, and then psychopy following the conda-relevant instructions on their website.

Cool. I got it to work yesterday. It's just that it's always such a headache. Could you do conda env export > environment.yml?

On the latter part, i also use different settings on macbook and linux. I also thought about different branches first. In the end my simple solution was having different setting files in my experiment repository. and then pick the relevant one when i run the experiment from the command line.

So you manually select the settings file and throw it into the run command or something?

lukassnoek commented 5 years ago

I have some time next week Wednesday and Thursday in between scans, so we can discuss these things. But to shortly comment on your points:

  1. It's reasonable to require Python3 for exptools2, especially since pylink has been ported to Py3.6
  2. I had little trouble installing (the latest version of) psychopy (for Py3) on both Linux and Mac. No clue about Py2 though. Regardless, making a conda environment/channel sounds like a good idea.
  3. The yaml-settings thing now also works with "overloading", where there is set of default settings (data/default_settings.yml), which can be overridden by a custom settings-file. Exptools2 will fall back on the default settings for any setting not defined in your custom settings file. W.r.t. flexibility in different testing environments: I'm doing the same thing as Marco, creating different settings-files for different testing envs, like settings_lab_uva.yml and settings_spinoza_scanner.yml, and settings_spinoza_testruimte.yml, which can be automatically selected based on the hostname, e.g.:
import socket
if socket.gethostname() == 'uva_108':
    settings = 'settings_lab_uva.yml'
else:
    settings = 'settings_spinoza_scanner.yml'

I'm also fine with hardcoding some "default" environments in exptools2 in the data/ directory.

Gilles86 commented 5 years ago

I installed exptools2 normally with setup.py -develop, and then psychopy following the conda-relevant instructions on their website.

Great, but they install python 3.5, which makes exptools2 not work, since exptools2 uses this super fancy new python 3.6 string formatting:

  File "/Users/gilles/miniconda3/envs/psypy3/lib/python3.5/site-packages/exptools2-0.1.dev0-py3.5.egg/exptools2/core/session.py", line 91
    raise IOError(f"Settings-file {self.settings_file} does not exist!")

So, @marcoaqil , what did you do exactly?

lukassnoek commented 5 years ago

I used python 3.6 so far, also because the pylink builds are geared towards 3.6. If you guys think we should accommodate all 3.x versions, we should drop the f-strings (fine with me).

Gilles86 commented 5 years ago

I used python 3.6 so far, also because the pylink builds are geared towards 3.6. If you guys think we should accommodate all 3.x versions, we should drop the f-strings (fine with me).

I think it's fine to use python 3.6, but it should be easy for users to install your package.

I slightly edited the installation 'script' of psychopy documentation to the following:

conda create -n psypy3 python=3.6
conda activate psypy3
conda install numpy scipy matplotlib pandas pyopengl pillow lxml openpyxl xlrd configobj pyyaml gevent greenlet msgpack-python psutil pytables requests[security] cffi seaborn wxpython cython pyzmq pyserial
conda install -c conda-forge pyglet pysoundfile python-bidi moviepy pyosf
pip install zmq json-tricks pyparallel sounddevice pygame pysoundcard psychopy_ext psychopy
pip install git+https://github.com/VU-Cog-Sci/exptools2/

Experiments run, but when I prematurely end them using q, Iget a new error:

python main.py                                                                               ✔  10641  10:18:41
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
/Users/gilles/miniconda3/envs/psypy3/lib/python3.6/site-packages/exptools2/core/eyetracker.py:19: UserWarning: Pylink is not installed! Eyetracker cannot be used
  warnings.warn(msg)
Traceback (most recent call last):
  File "main.py", line 122, in <module>
    session = FlickerSession('sub-01', settings_file=settings)
  File "main.py", line 42, in __init__
    super().__init__(output_str, output_dir=None, settings_file=settings_file)
  File "/Users/gilles/miniconda3/envs/psypy3/lib/python3.6/site-packages/exptools2/core/session.py", line 72, in __init__
    self.win = self._create_window()
  File "/Users/gilles/miniconda3/envs/psypy3/lib/python3.6/site-packages/exptools2/core/session.py", line 129, in _create_window
    t_per_frame = 1. / self.actual_framerate
TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'

Otherwise, I get the matplotlib error.

python main.py                                                                    ABRT(-6) ↵  10656  10:23:44
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
/Users/gilles/miniconda3/envs/psypy3/lib/python3.6/site-packages/exptools2/core/eyetracker.py:19: UserWarning: Pylink is not installed! Eyetracker cannot be used
  warnings.warn(msg)
Starting trial 0
    Phase 0 start: 0.01645

Duration experiment: 0.416

2019-05-27 10:24:01.015 python[84617:390659] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7fa1dd750580
2019-05-27 10:24:01.017 python[84617:390659] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7fa1dd750580'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff304be43d __exceptionPreprocess + 256
    1   libobjc.A.dylib                     0x00007fff5c3cb720 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff3053b255 -[NSObject(NSObject) __retain_OA] + 0
    3   CoreFoundation                      0x00007fff3045dad0 ___forwarding___ + 1486
    4   CoreFoundation                      0x00007fff3045d478 _CF_forwarding_prep_0 + 120
    5   libtk8.6.dylib                      0x0000000118c2331d TkpInit + 413
    6   libtk8.6.dylib                      0x0000000118b7b17e Initialize + 2622
    7   _tkinter.cpython-36m-darwin.so      0x00000001189a3a16 _tkinter_create + 1174
    8   python                              0x0000000107105068 _PyCFunction_FastCallDict + 200
    9   python                              0x00000001071da61f call_function + 143
    10  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    11  python                              0x00000001071cb8c9 _PyEval_EvalCodeWithName + 425
    12  python                              0x00000001071db2cc _PyFunction_FastCallDict + 364
    13  python                              0x0000000107083f80 _PyObject_FastCallDict + 320
    14  python                              0x00000001070ab5f8 method_call + 136
    15  python                              0x000000010708b5ce PyObject_Call + 62
    16  python                              0x000000010712c5b5 slot_tp_init + 117
    17  python                              0x0000000107130af1 type_call + 241
    18  python                              0x0000000107083ef1 _PyObject_FastCallDict + 177
    19  python                              0x000000010708c137 _PyObject_FastCallKeywords + 327
    20  python                              0x00000001071da718 call_function + 392
    21  python                              0x00000001071d8225 _PyEval_EvalFrameDefault + 47013
    22  python                              0x00000001071da9dc fast_function + 188
    23  python                              0x00000001071da67c call_function + 236
    24  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    25  python                              0x00000001071cb8c9 _PyEval_EvalCodeWithName + 425
    26  python                              0x00000001071db2cc _PyFunction_FastCallDict + 364
    27  python                              0x0000000107083f80 _PyObject_FastCallDict + 320
    28  python                              0x00000001070ab5f8 method_call + 136
    29  python                              0x000000010708b5ce PyObject_Call + 62
    30  python                              0x00000001071d8376 _PyEval_EvalFrameDefault + 47350
    31  python                              0x00000001071cb8c9 _PyEval_EvalCodeWithName + 425
    32  python                              0x00000001070ce92d function_call + 125
    33  python                              0x000000010708b5ce PyObject_Call + 62
    34  python                              0x00000001071d8376 _PyEval_EvalFrameDefault + 47350
    35  python                              0x00000001071cb8c9 _PyEval_EvalCodeWithName + 425
    36  python                              0x00000001071daa8a fast_function + 362
    37  python                              0x00000001071da67c call_function + 236
    38  python                              0x00000001071d8225 _PyEval_EvalFrameDefault + 47013
    39  python                              0x00000001071da9dc fast_function + 188
    40  python                              0x00000001071da67c call_function + 236
    41  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    42  python                              0x00000001071da9dc fast_function + 188
    43  python                              0x00000001071da67c call_function + 236
    44  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    45  python                              0x00000001071da9dc fast_function + 188
    46  python                              0x00000001071da67c call_function + 236
    47  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    48  python                              0x00000001071da9dc fast_function + 188
    49  python                              0x00000001071da67c call_function + 236
    50  python                              0x00000001071d8175 _PyEval_EvalFrameDefault + 46837
    51  python                              0x00000001071cb8c9 _PyEval_EvalCodeWithName + 425
    52  python                              0x000000010722455c PyRun_FileExFlags + 252
    53  python                              0x0000000107223a34 PyRun_SimpleFileExFlags + 372
    54  python                              0x000000010724a7c6 Py_Main + 3734
    55  python                              0x000000010707bf59 main + 313
    56  libdyld.dylib                       0x00007fff5d499085 start + 1
    57  ???                                 0x0000000000000002 0x0 + 2
)
[1]    84617 abort
Gilles86 commented 5 years ago

OK so the matplotlib error can be solved by installing qt and pyqt using the following installation pattern:

conda create -n psypy3 python=3.6
conda activate psypy3
conda install numpy scipy matplotlib pandas pyopengl pillow lxml openpyxl xlrd configobj pyyaml gevent greenlet msgpack-python psutil pytables requests[security] cffi seaborn wxpython cython pyzmq pyserial qt pyqt
conda install -c conda-forge pyglet pysoundfile python-bidi moviepy pyosf
pip install zmq json-tricks pyparallel sounddevice pygame pysoundcard psychopy_ext psychopy
pip install git+https://github.com/VU-Cog-Sci/exptools2/
Gilles86 commented 5 years ago

See https://github.com/VU-Cog-Sci/exptools2/pull/17