search

klabhub / neurostim

Design and run visual neuroscience experiments using Matlab and the Psychophysics Toolbox.
MIT License
5 stars 3 forks source link

SOFTWARE-OVERLAY screen type providing poor man's VPixx M16 functionality. #107

Closed cnuahs closed 5 years ago

cnuahs commented 5 years ago

Implementation of a SOFTWARE-OVERLAY screen type, replicating (in software, using a custom shader) the dual CLUT overlay of the VPixx M16 mode.

If you don't have a VPixx this gives you some of the same features, specifically the ability to mirror the subject's display to a second 'console' display, with separate CLUT for subject and console displays. This allows one to overlay information (eye position, fixation windows etc.) on the console display that is then visible only to the experimenter. It does this using a custom shader so the overhead (of the mirroring) is close to zero.

You can go the other way too, for example, rendering something like the diode square on the subject's display (where your photodiode is) but not on the console display.

I've been using this for >6 mths now. I think it works well.

The only point of potential conflict is probably in cic's updateOverlay() method... which is also used by the 'VPIXX-M16' screen type. I think this is clean, but I don't have a VPixx so cannot confirm it doesn't break anything there.

@bartkrekelberg, maybe someone is your lab can test this branch and make sure it doesn't break your use of the VPixx M16 mode.

jonaskn commented 5 years ago

Just a quick comment that the reason this uses indexed colors was for maximized compatibility when used as a replacement for the VPIXX overlay. If it's just used for adding information for the console display, one could easily change this to use regular full color of the overlay instead. jonas

cnuahs commented 5 years ago

Actually, @bartkrekelberg @adammorrissirrommada, hold off on this. Clearing the overlay in cic.beforeBlock() isn't working properly with the new FSM'ified master branch. Seems to be related to the new .textWindow and clearOverlay() business.

The draw origin when beforeBlock() is called is the top-left of screen, so cic.overlayRect needs to be [0 0 xpixels ypixels], not [-xpixels/2 -ypixels/2, xpixels/2 ypixels/2]... which seems to be required to clear the overlay properly during a trial (i.e., after each frame) otherwise it only clears the upper right quadrant.

This needs further investigation, I'm not sure what is going on here.

bartkrekelberg commented 5 years ago

I noticed that too on the VPixx ... the overlay is not clearing properly and I am not sure why.

It was on my list to debug, but that is hard to do remotely, so it has been on the todo list for a while...

( We started using white fixation dots instead!)

B

From: Shaun Cloherty notifications@github.com Sent: Sunday, December 16, 2018 9:26:30 PM To: klabhub/neurostim-ptb Cc: Bart Krekelberg; Mention Subject: Re: [klabhub/neurostim-ptb] SOFTWARE-OVERLAY screen type providing poor man's VPixx M16 functionality. (#107)

Actually, @bartkrekelberghttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbartkrekelberg&data=02%7C01%7Cbart%40rutgers.edu%7C2604286185674eba155108d663c7126e%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806104003015099&sdata=zBj34f8iLHQVhBUalz%2BfDpR7gxyzJk3pa%2Fw%2BECk0sSc%3D&reserved=0 @adammorrissirrommadahttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fadammorrissirrommada&data=02%7C01%7Cbart%40rutgers.edu%7C2604286185674eba155108d663c7126e%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806104003015099&sdata=0hSQLif9tzhh%2B888N13BNuwRCOwsqYUGz6U3MTlq%2FWw%3D&reserved=0, hold off on this. Clearing the overlay in cic.beforeBlock() isn't working properly with the new FSM'ified master branch. Seems to be related to the new .textWindow and clearOverlay() business.

The draw origin when beforeBlock() is called top-left of screen, so cic.overlayRect needs to be [0 0 xpixels ypixels], not [-xpixels/2 -ypixels/2, xpixels/2 ypixels/2]... which seemed to be required to clear the overlay properly during a trial (otherwise it only clears the upper right quadrant).

This needs further investigation, I'm not sure what is going on here.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fklabhub%2Fneurostim-ptb%2Fpull%2F107%23issuecomment-447703845&data=02%7C01%7Cbart%40rutgers.edu%7C2604286185674eba155108d663c7126e%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806104003015099&sdata=sazTcDLcDff0OVDB%2Bc6IPxOAP7Dl8L4fnCoT1TSSq%2BA%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALbI09nCnpzsjRLtK3d2_iouTy96ooV4ks5u5wDWgaJpZM4ZVGFa&data=02%7C01%7Cbart%40rutgers.edu%7C2604286185674eba155108d663c7126e%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806104003015099&sdata=beCK1ZS51IpxBXbv4D3JIXxQ2AjWWAyt7jrxHg3iPpo%3D&reserved=0.

cnuahs commented 5 years ago

My problem with the overlay stems from the fact that the plugin class calls Screen('glTranslate'...) etc. to set the draw origin to the centre of the screen, before calling the beforeFrame() method. As a result, the overlay would only be cleared properly up until the first call to beforFrame(), after that it would only clear one quadrant of the screen.

The fix here (but there may be a better one) pushes the current (default?) matrix before calling Screen('glTranslate',...), beforFrame() etc. in plugin.base(), and then pops it afterward.

Calls to clearOverlay()/Screen('FillRect',...) in the frame loop then behave as expected.

@bartkrekelberg I'd be interested to know if this also addresses your issue when using the VPixx, or if that is a separate problem.

bartkrekelberg commented 5 years ago

I will test it next time I'm in the lab (Mid jan)

B

Bart Krekelberg, PhD

Professor, Rutgers University - Newark

Co-Director, Center for Molecular and Behavioral Neuroscience

Associate Director, Rutgers University Brain Imaging Center

197 University Avenue

Newark, NJ 07102

T: 973 419 5118

E: bart@rutgers.edu

W: vision.rutgers.edu

From: Shaun Cloherty notifications@github.com Sent: Monday, December 17, 2018 8:10:44 PM To: klabhub/neurostim-ptb Cc: Bart Krekelberg; Mention Subject: Re: [klabhub/neurostim-ptb] SOFTWARE-OVERLAY screen type providing poor man's VPixx M16 functionality. (#107)

My problem with the overlay stems from the fact that the plugin class calls Screen('glTranslate'...) etc. to set the draw origin to the centre of the screen, before calling the beforeFrame() method. As a result, the overlay would only be cleared properly up until the first call to beforFrame(), after that it would only clear one quadrant of the screen.

The fix here (and the may be a better way) pushes the current (default?) matrix before calling Screen('glTranslate',...), beforFrame() etc. in plugin.base(), and then pops it afterward.

Calls to clearOverlay()/Screen('FillRect',...) in the frame loop then behave as expected.

@bartkrekelberghttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbartkrekelberg&data=02%7C01%7Cbart%40rutgers.edu%7Cea852817f73e424ffdbb08d66485a665%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806922516549740&sdata=fDbuXQrmhhynArdGlScaFO2qhI7BPPN4vEahOJ6TwtY%3D&reserved=0 I'd be interested to know if this also addresses your issue when using the VPixx, or if that is a separate problem.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fklabhub%2Fneurostim-ptb%2Fpull%2F107%23issuecomment-448058801&data=02%7C01%7Cbart%40rutgers.edu%7Cea852817f73e424ffdbb08d66485a665%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806922516549740&sdata=JypgV%2FG6lkMsJMTpfB0oNI0Y%2BtcZJmYlbi0E4TtV2tA%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALbI06WETgAhfDLXmDqQnvQt6VhhW4Mcks5u6ECUgaJpZM4ZVGFa&data=02%7C01%7Cbart%40rutgers.edu%7Cea852817f73e424ffdbb08d66485a665%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636806922516549740&sdata=1Uw0N%2BgyZFRhz83OqhXtEUpkCD3ohNWzGvTzJdo3Da0%3D&reserved=0.

bartkrekelberg commented 5 years ago

This seems to work fine, but one extra glLoadIdentity is needed to make sure the reference frame is shifted back to the top left corner before clearing with the overlayRect. (An alternative would be to "correct" the overlayRect, but given that it is in pixels and not screen units, I think the loadIdentity in the base() funciton is prepared. I will merge and add this.