joystick-compatibility

[ahmad11111111111]

Objective Setting up the controller as a device for the BCI system to sample data from the controller

Actions -formatting the application so its installable as a python library -Getting the application to record controller data from a game controller

• Converting the data recording process into a asynchronous background process
• ensuring the process can start and end successfully without a deadlock occurring
• formatting the application so that the Libbici application can call and sample the data with the ECoG data.

[ahmad11111111111]

-toml file seems to be working as intended, but the main branch of nexus is listed as a dependancy so the gitlab stuff won't work.

-i think I got what you meant with the get_input function, but I'd appreciate it if you could look over the ipynb file (will eventually convert to .py) and let me know if I'm on the right track when you get a chance.

[ahmad11111111111]

Also I'm flying into Jearsy to drive my girlfriend down to Miami Monday and I probably won't be back till late Thursday night, so we can postpone a zoom. Once class starts up, I'm going to make a habit of coming in lab MWF after my morning class, so I can get the ball rolling on this project.

[ahmad11111111111]

whenever you get the chance can you look over this code and let me know if this is the right approach/what else needs to be done.

[ahmad11111111111]

class GameControllerInput(InputDevice):
    def __init__(
        self,
        port: Any,
        char_controllers: List[CharController],
        log_path: Optional[Path] = None,
    ):
        super().__init__(port, char_controllers, log_path)
        pygame.init()

        # Set up the game controller
        self.controller = pygame.joystick.Joystick(0)
        self.controller.init()

        # Open a file for recording the inputs
        self.file_path = log_path or Path("controller_inputs.txt")
        self.file = open(self.file_path, "w")

    def get_input(self) -> None:
        try:
            while True:
                # Check for events
                for event in pygame.event.get():
                    # Check if the event is a controller input
                    if event.type == pygame.JOYBUTTONDOWN or event.type == pygame.JOYAXISMOTION:
                        # Get the input details
                        input_type = ""
                        input_value = ""

                        if event.type == pygame.JOYBUTTONDOWN:
                            input_type = "Button"
                            input_value = event.button
                        elif event.type == pygame.JOYAXISMOTION:
                            input_type = "Axis"
                            input_value = (event.axis, event.value)

                        # Write the input to the file
                        self.file.write(f"{input_type}: {input_value}\n")

                # Exit the loop if the controller's 'A' button is pressed
                if self.controller.get_button(0) == 1:
                    break
        finally:
            # Close the file and quit pygame
            self.file.close()
            pygame.quit()

    def read_recorded_inputs(self) -> str:
        # Read and return the recorded inputs
        with open(self.file_path, "r") as file:
            inputs = file.read()
            return inputs

[ahmad11111111111]

I formated it inside class GameControllerInput that directly inherits from InputDevice. I also added the get input functionality to the get-input function.

[kevincar]

@ahmad11111111111 Not sure if you still need me to look at this since the issue is marked as solved.

I guess if it works it works. Have you tested it?

A couple possible issues I see though:

1. Line 22. get_input() will never shut down properly because it is an unconditional loop. The only way to stop that method is to crash the program.
2. By using InputDevice it looks like you're hoping to use our libbci system. If this is the case, then get_input() should return a decoded value of data that the system can use. get_input() should return an integer representing decoded input You can simply have it return 0 since we're not interested in decoding signal from a Joystick. A second problem is that your infinite loop will cause the system to halt, and you won't get data from the nexus simultaneously recorded.
3. Writing all data to a file neglects time. We have no way of telling at what time each data point was collected, and therefore don't have a way to match up the joy stick data with the ECoG data. It may also be a good idea to get a baseline sampling rate (i.e., how fast does python/pygame and your controller sample data from the joystick). We can also throttle the sampling and downsample to a slightly lower time frame to force a sampling rate we can manage.
4. The pygame object is initialized during __init__, but is destroyed/quit during get_input(). This is an issue. It means that you only anticipate get_input() to be called once? In the same way you can call get_input() on the nexus or even get_raw_data() and see the raw data values, you should strive to achieve the same effect with the joystick class.

One way to solve this is by separating out areas of concern. You have 3 areas of concern: 1) Sampling data from the joystick, 2) Interfacing with the joystick, and 3) storing joystick data. For managing the pygame and joystick data, you can create a JoyStick class that handles all of the joystick data sampling. This may need to be implemented using subprocesses so that the joy stick can work on it's own. This is similar to how we implemented the NexusInstrument class. It can sample the data and store it to an internal buffer. The InputDevice class can then be concerned about hooking into the BCI system. The BCI System will call get_input on all devices as fast as it can. This means each time get_input is called you should call a method on the JoyStick class to collect all packets of data collected on the JoyStick from the last time get_input was called and flush the buffer.

However, if you code is working then I'd say continue with that and we'll circle back

[ahmad11111111111]

Thanks. I wasn't sure how the libbci system worked, I have a good idea now. I'll try and get something working before our meeting.

---

From: Kevin Davis @.> Sent: Monday, August 21, 2023 7:10:56 PM To: ahmad11111111111/Brain-Controller-App @.> Cc: Al-Salem, Ahmad Tarik @.>; Mention @.> Subject: [EXTERNAL] Re: [ahmad11111111111/Brain-Controller-App] nexus-compatibility (Issue #1)

CAUTION: This email originated from outside the organization. DO NOT CLICK ON LINKS or OPEN ATTACHMENTS unless you know and trust the sender.

@ahmad11111111111https://github.com/ahmad11111111111 Not sure if you still need me to look at this since the issue is marked as solved.

I guess if it works it works. Have you tested it?

A couple possible issues I see though:

1. Line 22. get_input() will never shut down properly because it is an unconditional loop. The only way to stop that method is to crash the program.
2. By using InputDevice it looks like you're hoping to use our libbci system. If this is the case, then get_input() should return a decoded value of data that the system can use. get_input() should return an integer representing decoded input You can simply have it return 0 since we're not interested in decoding signal from a Joystick. A second problem is that your infinite loop will cause the system to halt, and you won't get data from the nexus simultaneously recorded.
3. Writing all data to a file neglects time. We have no way of telling at what time each data point was collected, and therefore don't have a way to match up the joy stick data with the ECoG data. It may also be a good idea to get a baseline sampling rate (i.e., how fast does python/pygame and your controller sample data from the joystick). We can also throttle the sampling and downsample to a slightly lower time frame to force a sampling rate we can manage.
4. The pygame object is initialized during init, but is destroyed/quit during get_input(). This is an issue. It means that you only anticipate get_input() to be called once? In the same way you can call get_input() on the nexus or even get_raw_data() and see the raw data values, you should strive to achieve the same effect with the joystick class.

One way to solve this is by separating out areas of concern. You have 3 areas of concern: 1) Sampling data from the joystick, 2) Interfacing with the joystick, and 3) storing joystick data. For managing the pygame and joystick data, you can create a JoyStick class that handles all of the joystick data sampling. This may need to be implemented using (subprocesses)[https://docs.python.org/3/library/subprocess.html] so that the joy stick can work on it's own. This is similar to how we implemented the NexusInstrument classhttps://github.com/umnil/device-nexus/blob/main/nexus/nexus_instrument.py. It can sample the data and store it to an internal buffer. The InputDevice class can then be concerned about hooking into the BCI system. The BCI System will call get_input on all devices as fast as it can. This means each time get_input is called you should call a method on the JoyStick class to collect all packets of data collected on the JoyStick from the last time get_input was called and flush the buffer.

However, if you code is working then I'd say continue with that and we'll circle back

— Reply to this email directly, view it on GitHubhttps://github.com/ahmad11111111111/Brain-Controller-App/issues/1#issuecomment-1687180959, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZ66B47J7IFLVVAWVUU3EU3XWPTIBANCNFSM6AAAAAA3ONCVHI. You are receiving this because you were mentioned.Message ID: @.***>