Cleanest way to log additional stimulus information.

[goraj]

Hi,

I would like to log additional information for an experiment. A good example would be the stimulus color and direction that is presented to the fish.

I tried adding two dynamic columns to dynamic_parameters in my stimulus class, that way I can use these columns to store some information as float in df_param.

Unfortunately this does not seem to work correctly and it seems not very clean to me as it ends up in np.interp. It would be great if you could suggest the best way to log information/conditions like this.

I attached a small example that tries to log color and direction using the df_param.

from collections import namedtuple

from stytra import Stytra, Protocol
from stytra.stimulation.stimuli.visual import Pause, FullFieldVisualStimulus, PaintGratingStimulus
from stytra.stimulation.stimuli import MovingGratingStimulus, InterpolatedStimulus
from lightparam import Param
import pandas as pd
import numpy as np
from pathlib import Path

from enum import IntEnum
class Direction(IntEnum):
    LEFT = 0
    RIGHT = 1

class Color(IntEnum):
    MILDGREY = 0
    GREY = 1
    SUPERGREY = 2

class MovingGratingStimulusTest(PaintGratingStimulus, InterpolatedStimulus):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.dynamic_parameters.append("x")

        self.dynamic_parameters.append("direction")
        self.dynamic_parameters.append("stimulus_color")

# 1. Define a protocol subclass
class GratingsProtocol(Protocol):
    name = "gratings_protocol"

    video_path = str(Path(r'C:\Users\user\PycharmProjects\stytra-private\stytra\examples\assets\fish_compressed.h5'))
    print(video_path)
    stytra_config = dict(
        tracking=dict(method="tail", estimator="vigor"),
        camera=dict(
            video_file=video_path
        ),
        # Replace this example file with the desired camera config, such as
        # camera_config = dict(type="ximea")
        # for a ximea camera, etc. Not needed if the setup already has the
        # # stytra_setup_config.json file
        # camera_config=dict(
        #     video_file=r"J:\_Shared\stytra\fish_tail_anki.h5"
        # ),
    )

    def __init__(self):
        super().__init__()
        self.t_pre = Param(1.)  # time of still gratings before they move
        self.t_move = Param(1.)  # time of gratings movement
        self.grating_vel = Param(-10.)  # gratings velocity
        self.grating_period = Param(10)  # grating spatial period
        self.grating_angle_deg = Param(90.)  # grating orientation
        self.num_repetitions = Param(1)  # number of repetitions

    def get_stim_sequence(self):
        # Use six points to specify the velocity step to be interpolated:
        t_phase = np.array([
            0,
            self.t_pre,
            self.t_pre,
            self.t_pre + self.t_move,
            self.t_pre + self.t_move,
            2 * self.t_pre + self.t_move,
        ])
        t = []

        # create list with time points extended by length num.repetition
        for iteration in range(self.num_repetitions):
            t.extend(
                list(
                    t_phase + t_phase[-1] * iteration
                )
            )

        vel_phase = [0, 0, self.grating_vel, self.grating_vel, 0, 0]
        vel = []
        vel.extend(vel_phase * self.num_repetitions)
        print(f't: {t}')
        print(f'vel: {vel}')
        #col = [(255, 255, 255)] * len(t)

        df = pd.DataFrame(dict(t=t, vel_x=vel))
        stimulus_sequence = []

        stimulus_param = namedtuple(
            'StimulusParam',
            [
                'color',
                'grating_angle',
                'stimulus_color',
                'direction',
            ],
            verbose=True
        )

        parameters = [
            stimulus_param(
                stimulus_color=Color.MILDGREY,
                direction=Direction.RIGHT,
                color=(40,) * 3,
                grating_angle=90
            ),
            stimulus_param(
                stimulus_color=Color.GREY,
                direction=Direction.LEFT,
                color=(100,) * 3,
                grating_angle=270
            ),
            stimulus_param(
                stimulus_color=Color.SUPERGREY,
                direction=Direction.LEFT,
                color=(155,) * 3,
                grating_angle=270
            ),
        ]
        for param in parameters:
            df['direction'] = float(param.direction)
            df['stimulus_color'] = float(param.stimulus_color)

            stimulus_sequence.append(
                MovingGratingStimulusTest(
                    df_param=df,
                    grating_col_1=param.color,
                    grating_angle=param.grating_angle * np.pi / 180,
                    # self.grating_angle_deg * np.pi / 180,
                    grating_period=self.grating_period,
                ),
            )
        return stimulus_sequence

if __name__ == "__main__":
    # This is the line that actually opens stytra with the new protocol.
    st = Stytra(protocol=GratingsProtocol())

[vigji]

All the "static" information of a stimulus (i.e., everything that can be passed as a kw argument and does not have to be interpolated) get logged into the "log" field of the json experiment metadata. For such quantities, which remain constant for each stimulus object, you don't need a column in the dataframe. After running your script, open the json file as a dictionary, and look into the stimulus/log field:

import json
with open(r"C:\Users\lpetrucco\Desktop\gratings_protocol\190628_f0\134825_metadata.json", "r") as f:
    metadata_dict = json.load(f)
metadata_dict["stimulus"]["log"]

This will be a list containing all the static attributes of you stimuli sequence. After running your script, I have a 3 elements list, each element being a dictionary with a "color" field containing the value that you passed to the stimulus class as the "grating_col_1" argument (I only now notice this naming inconsistency, that the "grating_col_1" argument is actually passed as the "color" field in the stimulus. we will maybe fix this in the future).

Note that in your code the df is not doing anything to the stimulus color, and the code is working only because you are passing it as a static argument. For the dataframe to effectively act on the stimulus properties, its columns need to be named exactly as the stimulus property that should be changing, in this case "color" instead of "stimulus_color".

Sorry if things were not clear - we will add soon a more detailed section in the documentation!

[goraj]

Indeed the class attributes are not consistent with the passed kwargs in some cases, ie. direction/angle ends up as theta. This was a little bit confusing. Thank you for clearing that up!

[vilim]

I'll reopen this issue as we need to make the stimulus construction and documentation more consistent.