KeyError When Reading .seq File generated from MATLAB pulseq 1.4.0 in pypulseq 1.4.0

[snawarhussain]

Description

Description

I encountered a KeyError when attempting to read a .seq file in pypulseq (compatible with pulseq 1.4.0) generated by MATLAB Pulseq version 1.4.0. Here is the error message I received:

packages\pypulseq\Sequence\sequence.py:987, in Sequence.read(self, file_path, detect_rf_use)
    977 def read(self, file_path: str, detect_rf_use: bool = False) -> None:
    978     """
    979     Read `.seq` file from `file_path`.
    980 
   (...)
    985         Path to `.seq` file to be read.
    986     """
--> 987     read(self, path=file_path, detect_rf_use=detect_rf_use)

File pypulseq\Sequence\read_seq.py:325, in read(self, path, detect_rf_use)
    314         old_data = np.array([amplitude, grad.shape_id, grad.delay])
    315     new_data = np.array(
    316         [
    317             amplitude,
   (...)
    323         ]
    324     )
--> 325     self.grad_library.update_data(amplitude_ID, old_data, new_data, "g")
    326 else:
    327     grad_prev_last[j] = 0

File pypulseq\event_lib.py:256, in EventLibrary.update_data(self, key_id, old_data, new_data, data_type)
    241 def update_data(
    242     self,
    243     key_id: int,
   (...)
    246     data_type: str = str(),
    247 ):
    248     """
    249     Parameters
    250     ----------
   (...)
    254     data_type : str
    255     """
--> 256     self.update(key_id, old_data, new_data, data_type)

File pypulseq\event_lib.py:237, in EventLibrary.update(self, key_id, old_data, new_data, data_type)
    235     else:
    236         key = tuple(old_data)
--> 237     del self.keymap[key]
    239 self.insert(key_id, new_data, data_type)

KeyError: (1021859.496, 1.0, 0.0, 0.0

To Reproduce

1. generate a .seq file by MATLAB Pulseq 1.4.0.
2. Attempt to read the file using pypulseq's Sequence().read('file.seq') method.
3. Encounter the KeyError.

Expected behavior I expected the .seq file to be read without any errors and for the sequence to be loaded into the Sequence object just like a .seq generated by pypulseq would be read.

Desktop:

• pypulseq version: [1.4.0]
• Python version: [3.11.5]
• Operating System: [Windows 11 OS]

Additional context I've verified that the .seq file is not corrupted and it is read by the Matlab pulseq. On the other hand, The .seq files generated by pypulseq are readable by pypulseq with the same method. This issue seems specific to files generated by MATLAB Pulseq 1.4.0. when you try to read them using pypulseq.

[btasdelen]

@snawarhussain Thanks for the bug report. I reproduced the error on my part. I am looking into it.

[btasdelen]

Small updates on the bug. This issue does not happen with all Matlab generated .seq files. I tried several examples, only one of them triggered the bug, so I have yet to find the actual culprit. Meanwhile, I tracked down the issue down to event_lib.py. What happens is, extended trapezoids are first added to the library without the first and last properties inside read function:

                amplitude_ID = event_idx[j + 2]
                amplitude = self.grad_library.data[amplitude_ID][0]
                if version_combined >= 1004000:
                    old_data = np.array(
                        [amplitude, grad.shape_id, grad.time_id, grad.delay]
                    )
                else:
                    old_data = np.array([amplitude, grad.shape_id, grad.delay])
                new_data = np.array(
                    [
                        amplitude,
                        grad.shape_id,
                        grad.time_id,
                        grad.delay,
                        grad.first,
                        grad.last,
                    ]
                )
                self.grad_library.update_data(amplitude_ID, old_data, new_data, "g")

Then, they are "updated" by adding those two properties to the key, but actually the old key is just deleted and a new entry is created with the new key. But, for some reason this is repeated for the same event later on, but as the old key is already deleted, it can not be deleted again and an exception occurs right here in update function:

        if len(self.keys) >= key_id:
            if self.numpy_data:
                old_data = np.asarray(old_data)
                key = old_data.tobytes()
            else:
                key = tuple(old_data)
            del self.keymap[key] # <---- Exception

To me, this should cause an error for every .seq file. I am trying to understand what triggers the fail, though.

@snawarhussain Is it possible for you to share what .seq file triggers the error and what equivalent Python .seq file does not?

[btasdelen]

I think I got the bug. It is trigger when an extended trapezoid with same shape_id is played on the same block at different channels. The way update works, it gets the block first, then iterates over each grad channel, and updates the event library if it finds an "old" key. So the first channel gets updated, but as the block is acquired outside the loop, of course the next channel with the same shape_id stays the same, even though the event library is updated.

I will devise a fix and commit.

[btasdelen]

Solved in 90c84ed7b09700a59da4c6bbe500b573296d7ce2. Feel free to reopen if the issue is not solved for you.

[snawarhussain]

@btasdelen Thank you so much for your hard work, that was lightning fast!