problems with example1.ipynb

[m4liao]

Please let me know if this is not the right way to report an issue and/or I am just doing something wrong so I can delete this. I can run every single example notebook except for the first one (single-trial GLM estimation in a rapid event-related design), which coincidentally is the only one I am trying to understand. The 10th chunk where I run glmsingle_obj.fit(...) is where I get an error while cross-validating different numbers of regressors. This is the error I get (along with a screenshot):

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (12,) + inhomogeneous part.

Fitting type-B model (fithrf) works just fine. I have not yet tested the matlab scripts but I will when I get a chance.

[iancharest]

Hey, sorry for the delay in getting back to you about this, we were at a conference last week. I am not sure I understand what is going on. You are trying to run the code from the example demo with your own fMRI volumes and Design matrices, and get errors? It would be useful to know the shape of your fMRI Volumes and Design Matrices that you feed in to GLMsingle in order to try and understand what is happening. --

[m4liao]

Hi!

Hopefully I can just reply to this email directly. I am trying to run the example demo verbatim and I am running into that error. I can run every example except for python example 1 (event-related design). I didn't change anything at all. I did try it with my own fMRI volumes and design matrices, but it didn't work either. Thanks!

Best, Ming-Ray

On Thu, Jun 1, 2023 at 12:08 PM Ian Charest @.***> wrote:

Hey, sorry for the delay in getting back to you about this, we werw at a conference last week. I am not sure I understand what is going on. You are trying to run the code from the example demo with your own fMRI volumes and Design matrices, ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Hey, sorry for the delay in getting back to you about this, we werw at a conference last week. I am not sure I understand what is going on. You are trying to run the code from the example demo with your own fMRI volumes and Design matrices, and get errors? It would be useful to know the shape of your fMRI Volumes and Design Matrices that you feed in to GLMsingle in order to try and understand what is happening. --

— Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/cvnlab/GLMsingle/issues/90*issuecomment-1572453640__;Iw!!KwNVnqRv!D8wLE2K1rH5CN8j03wPCwto9lbLajqe4AHx0SeGoNIOj1-2TTStN5Wp-DKg3UCN1J4om_wfTD85vOS_0H1lO-dk-$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ARLDQPWV7VTFYT7OW52C6NLXJDEB3ANCNFSM6AAAAAAYIKK4BA__;!!KwNVnqRv!D8wLE2K1rH5CN8j03wPCwto9lbLajqe4AHx0SeGoNIOj1-2TTStN5Wp-DKg3UCN1J4om_wfTD85vOS_0Hw_AN6oX$ . You are receiving this because you authored the thread.Message ID: @.***>

-- Ming-Ray Liao any pronouns | PhD Candidate Psychological and Brain Sciences | Texas A&M University

[iancharest]

Hi, I am having trouble replicating this issue. Could you tell us a bit more, which version of python are you using, what OS are you running, etc, also, could you try and install the latest version of GLMsingle, e.g. pip install --ignore-installed git+https://github.com/cvnlab/GLMsingle.git

[m4liao]

Hi Ian,

I am on MacOS 10.13.6 (High Sierra) and I am using Python 3.10.2. I reinstalled GLMsingle and tried running example demo 1 again, but got the same result. I have changed absolutely nothing. Thanks!

Best, Ming-Ray [image: image.png]

On Fri, Jun 2, 2023 at 10:45 AM Ian Charest @.***> wrote:

Hi, I am having trouble replicating this issue. Could you tell us a bit more, which version of python are you using, what OS are you running, etc, also, could you try and install the latest version of GLMsingle, e. g. pip install --ignore-installed ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Hi, I am having trouble replicating this issue. Could you tell us a bit more, which version of python are you using, what OS are you running, etc, also, could you try and install the latest version of GLMsingle, e.g. pip install --ignore-installed git+https://github.com/cvnlab/GLMsingle.git

— Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/cvnlab/GLMsingle/issues/90*issuecomment-1573949180__;Iw!!KwNVnqRv!HmY4oiADmCFVwnGxOUHIyLIKsECiVvUbGzEgNUqAXgAjx1HRrpyctRI38n3YsfbRPcuumJicz1-zU43xu4UTLUUC$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ARLDQPUFDRQ2Q27M7HMNDCDXJIDB7ANCNFSM6AAAAAAYIKK4BA__;!!KwNVnqRv!HmY4oiADmCFVwnGxOUHIyLIKsECiVvUbGzEgNUqAXgAjx1HRrpyctRI38n3YsfbRPcuumJicz1-zU43xu1pTdV96$ . You are receiving this because you authored the thread.Message ID: @.***>

-- Ming-Ray Liao any pronouns | PhD Candidate Psychological and Brain Sciences | Texas A&M University

[alexanderdfung]

I am running into the same issue (on Python 3.9, fresh install of GLMSingle). The issue arises because validcolumns is a ragged list of 12 elements (the number of runs), with lengths 63, 62, 63, 62, ... etc. The demo was working for me in a previous version of GLMSingle (Feb 2021).

[kendrickkay]

We have having trouble reproducing your error. Can you post a screenshot and/or any other information?

[alexanderdfung]

Error message is identical to the first post. What is the intended shape of validcolumns that you get when it runs without error? I get a list of 12 elements with the following shapes: (63,) (62,) (63,) (62,) (63,) (62,) (63,) (62,) (63,) (62,) (63,) (62,) Because the shapes do not match, the concatenation command fails. Is this case supposed to be handled in calcbadness or should the elements of validcolumns always have the same shape?

[ryskina]

This happens to me as well whenever the number of voxels is not divisible by the chunk length. As a result, chunking.py creates a list of chunks of unequal lengths, and np.asarray() throws this error (https://github.com/cvnlab/GLMsingle/blob/main/glmsingle/utils/chunking.py#L37).

Creating arrays out of uneven sequences was allowed in earlier versions of Numpy though. I was able to run the code in chunking.py with Numpy 1.22.2 and got only this warning:

<stdin>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

But with Numpy 1.24.4 I get the error:

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

So maybe the solution is to downgrade to an earlier version of Numpy?

[jacob-prince]

Thank you @ryskina and @alexanderdfung for helping track down the source of the error. It sounds like Numpy versions are indeed the issue. I'll have this resolved as soon as possible.

[jacob-prince]

@ryskina and @alexanderdfung, I have addressed this issue by debugging the calcbadness.py file (https://github.com/cvnlab/GLMsingle/commit/573d90f7b949760e5d7fae9d5a1bae79d8ee10c1)

Issues with Example1.ipynb should be resolved now. please let me know if any issues persist with the example scripts.

[ryskina]

Thank you, @jacob-prince!

In our own implementation (not example1.ipynb) we were still experiencing the same issue at the chunking stage, even after the update:

Traceback (most recent call last):
  File "/nese/mit/group/evlab/u/ryskina/glmsingle/sandbox.py", line 138, in <module>
    results_glmsingle = glmsingle_obj.fit(
  File "/home/ryskina/miniconda3/envs/glmsingle/lib/python3.9/site-packages/glmsingle/glmsingle.py", line 515, in fit
    chunks = chunking(
  File "/home/ryskina/miniconda3/envs/glmsingle/lib/python3.9/site-packages/glmsingle/utils/chunking.py", line 37, in chunking
    return np.asarray(f)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (19,) + inhomogeneous part.

@ashleymalkin submitted a pull request addressing this: #99. With that, our code ran without issues.

[kendrickkay]

Thanks @ryskina !

[iancharest]

Hi, we are about to push a PR with a fix for the chunking.

We are having issues with the new fix to calcbadness, and we will fix this too in this PR. making everything an type object did not seem to solve it.

[ryskina]

Thank you @iancharest! We also ran into this with calcbadness, I suspected that it might be caused by the new fix but haven't had time to check yet:

File "/home/ryskina/miniconda3/envs/temp/lib/python3.9/site-packages/glmsingle/glmsingle.py", line 1100, in fit
    glmbadness[relix, :] = calcbadness(
  File "/home/ryskina/miniconda3/envs/temp/lib/python3.9/site-packages/glmsingle/ssq/calcbadness.py", line 75, in calcbadness
    mn = np.mean(results[0][:, whcol], axis=1)
IndexError: arrays used as indices must be of integer (or boolean) type

In the meantime, I've downgraded to numpy=1.22.2 and GLMSingle commit 3599fbc9b577b1b0fc44045afa786b3111109f99 and everything ran smoothly.

[jacob-prince]

we have merged changes to calcbadness.py and chunking.py to (hopefully) resolve this bug once and for all. closing this issue.

see: https://github.com/cvnlab/GLMsingle/pull/100