Problem with set_data

[ivankott]

Dear SCoT developers :) Recently, I've started exploring your toolbox for connectivity estimation in epilepsy EEG data. Thanks a bunch for your work! So, in a nutshell, from SCoT toolbox I only extract unmixing matrices and apply it later to my data in Matlab for dim reduction and estimation of different conditional entropies from sources. And somehow when I use the data which was set in the toolbox (save after ws.set_data) and I use the same data without passing through the toolbox I got different results. Moreover, the data is changing as well. Please find the screenshot in the following link. However, it's supposed to be the same, set_data should just stack all trials together. I'd really appreciate if you can help me resolve or understand this issue. Thanks in advance!

[cbrnr]

Could you share a minimal code snippet so that we can see how you created these lines?

[ivankott]

Sure, but the problem is in the data, not in the way how I visualize it.

figure();
X_scot = squeeze(data(7,:,:))'; %1 single trial from ws.set_data(data, readout_classes) .. data = ws.data_
yyaxis left;plot(X_scot(:,2)); %select only 1 channel to visualize
yyaxis right; plot(focal_data_filtered_for_scot.patient_name_1.signals(1).Beforeseizure(:,2)) %corresponding trial and channel from data before passing it through the toolbox

[cbrnr]

Yeah, what I meant was the code which creates the signals.

[ivankott]

))) sorry

def get_data_and_classes(input_dict=None, classes_to_get=[]):
    data = []
    classes = []
    for i, subject in enumerate(list(input_dict.keys())):
        for j, key in enumerate(list(subject_dict["sigs"].keys())):
            if key in classes_to_get:
                if subject_dict["sigs"][key]:
                    if key == "After_seizure":
                        temp_arr = np.transpose(
                            np.asarray(subject_dict["sigs"][key]))
                    elif key == "Before_seizure":
                        temp_arr = np.transpose(
                            np.asarray(subject_dict["sigs"][key]))
                    elif key == "Baseline":
                        temp_arr = np.transpose(
                            np.asarray(subject_dict["sigs"][key]))
                else:
                    continue

                result_arr = np.zeros((len(config.get("electrode_labels")), temp_arr.shape[1]))

                for label in subjects_electrode_labels:
                    if label in config.get("electrode_labels"):
                        # print("Found label:  {}".format(label))
                        label_index = config.get("electrode_labels").index(label)
                        result_arr[label_index, :] = temp_arr[label_index, :]

                #append to resulting array and classes array
                data.append(result_arr)
                classes.append(config.get("classes")[key])

    return np.asarray(data, dtype=np.float32), np.asarray(classes)

therefore I call this function:

......
classes_combinations = [('Baseline', 'Before_seizure')] 
for combination in classes_combinations:
    data, readout_classes = get_data_and_classes(input_dict=focal_data_filtered_for_scot, 
    classes_to_get=list(combination))
    .....
    ws = scot.Workspace({'model_order': order}, reducedim=reducedim, fs=Fs, locations=locs)
    ws.set_data(data, readout_classes)
    data = ws.data_
    ......
    matfile = 'focal_eeg_data_{}_{}.mat'.format(cl_name[0], cl_name[1])
    scipy.io.savemat(matfile, mdict={'data': data}, oned_as='row')

it's basically the way I save the data

[cbrnr]

I think I understand now what you mean. So you are saying that ws.data_ is different from data (which you use in ws.set_data)? If this is the case, can you send me these two arrays? Or if they are still identical maybe this happens during saving to .mat?

[ivankott]

Yes, that's what I mean. Thanks, I've just checked, and indeed this is happening during saving. I didn't even think this could give me such a strange problem. And I also couldn't find the reason for that in your code. Thanks a bunch!

[cbrnr]

I'd still be interested why this happened (it shouldn't)...