Selectable motion clouds

[RoboDoig]

Motion cloud textures will be selected from a list that is pre-loaded in TextureResources. Right now one must manually select the BindTexture source for the motion clouds. Would be good if the schema allowed named reference to the motion cloud texture selected on each block.

[ederancz]

This is part of the new contract.

[RoboDoig]

Note for me, check ranson-rig implementation for texture loading

[RoboDoig]

A related issue that will fall under this new implementation - currently the workflow assumes a hard-coded number of frames for each motion cloud. This value is used to 'wrap' the motion cloud sequence back around to the start during phase procession. The wrap value should be determined automatically or added as a parameter in the session settings. Ideally automatically.

[RoboDoig]

I've implemented the ability to select motion cloud textures by name in the schema and updated the local version of the Bonsai shaders package to allow automatic wrapping see here for longer term support for this feature.

There is still an error on the first motion cloud that is loaded, I think because I've implemented the GetTextureData operator as a source with the texture name as a parameter, somehow the initialisation order means that it tries to access the texture before the name property is updated.

[RoboDoig]

@EleonoraAmbrad - please see PR #85 for implementation of this. Seems to work but if you have a chance to test on one of your motion cloud experiments that would be great!

[EleonoraAmbrad]

Finally got to test this, it's working, but I have a few questions:

• I failed to load a .png file using Add image texture, does the input need to be a .mp4 file? I am asking because I am struggling really hard to generate an .mp4 file with the image size we need, but it seems to be straightforward with .png format so that would make it easier for me to start --> update: I managed, it was a matter of not setting number of frames to 1

• the mapping is now limited to 1 screen (might be an artifact from debugging?)

I am also wondering: I will need to display drifting clouds in open loop experiments, can I implement this using the current workflow? I would need to couple the movement of the cloud to a function generator producing a sine wave of different frequencies (mimicking temporal frequencies). Otherwise: will I need to generate the sinewave as csv, similar to what we do for the playback?

Thank you!

[EleonoraAmbrad]

Finally got to test this, it's working, but I have a few questions:

• I failed to load a .png file using Add image texture, does the input need to be a .mp4 file? I am asking because I am struggling really hard to generate an .mp4 file with the image size we need, but it seems to be straightforward with .png format so that would make it easier for me to start -->

update: I managed, it was a matter of not setting number of frames to 1

• the mapping is now limited to 1 screen (might be an artifact from debugging?)

update:solved, errpr on my side

I am also wondering: I will need to display drifting clouds in open loop experiments, can I implement this using the current workflow? I would need to couple the movement of the cloud to a function generator producing a sine wave of different frequencies (mimicking temporal frequencies). Otherwise: will I need to generate the sinewave as csv, similar to what we do for the playback?

update: just realised that the csv approach might not work because if i produce a sinewave it will not be harp timestamped, so I will probably try to generate a harp timestamped sinewave now

Thank you!

[EleonoraAmbrad]

sf_0=0.01-B_sf=0.06-theta=0-B_theta=0.25-alpha=0-seed=1

When trying to load this .mp4 file, which has the size we need (2 screens = 3840 frame size), I get an error saying: 'Index was outside the bounds of the array'. The error comes from the node BindTexture, within the DoBlock node. I tried changing the index to the one of the image, but it seems that it is automatically loadd (and therefore it cannot be read).

The file is under Textures, where the others are. Thanks for the help

[RoboDoig]

Issue was that last indexed frame on each block persisted to the next block in BindTexture. If therefore one loads a length 4096 texture and the last frame accessed is 4000; then one loads a length 256 texture the first index that the block will try and access is 4000 and out of bounds of the new texture. Fixed by always initialising new blocks to the 0th index of the texture sequence.

[RoboDoig]

Tested bugfixes, closing issue with #85