Missing data and go-nogo

[AlexanderFengler]

Provide likelihoods that can handle missing data and go-nogo paradigms.

Rough intro (with fluff) to the issue is here.

[moxtoby]

Hello Alex,

Is HSSM able to incorporate Go/No-go data? In HDDM, I used the following command to extract v, a, t and z for each participant:

GNGModel=hddm.HDDMStimCoding(mydata,depends_on={'v':'condition'},include='z', stim_col='condition',split_param='z', p_outlier=0.05) GNGModel.sample(20000, burn=2000)

Where rt for a withholding response is 999.

When I tried to input a single participant's GNG data in HSSM (updating correct and incorrect response as 1 and -1 respectively), using the following command, I hoped to get the v for go and no-go trials separately, a, t, and z:

J_gng_model = hssm.HSSM(data=df_t, model = 'ddm', hierarchical = False, loglik_kind = "blackbox", include=[{"name":"v", "prior":{"name":"Uniform", "lower":-8.0, "upper": 8.0}, "formula":"v~ 0+(1|condition)", },{"name":"a", "prior":{"name":"Uniform", "lower":0.2, "upper": 5.0},},{"name":"t", "prior":{"name":"Uniform", "lower":0.01, "upper":1.0},},],) J_gng_model.sample(draws=20000, tune=2000)

However, I am finding that even with 20000 samples, some of the parameters (specifically a and z) are not converging. For one of my participants, none of the parameters converged. When I increased the sample to 25000 then to 30000 another one of my participants, the RHat value actually increased rather than decreased. Please see the attached screenshot of the traces, and also the HDDM/HSSM comparison spreadsheet.

Is there another set of command options I should include in HSSM for it to work with Go/No-go data? I don't believe the parameters recovered by HSSM are equivalent to those in HDDM, especially the z parameter, which should be nearer the "go" boundary. Also, the "a" values in HSSM are not half of those in HDDM.

Many thanks again for your time and your help.

Best wishes, Michelle

Compare HDDM HSSM 3.xlsx

[LindaFiorini]

Hello Michelle,

I am new to DDM and to this library so I am having some issues in fitting my Go/No-Go in HSSM. Could you solve your issue eventually? In case the answer is yes, would you share the code?

Thank you very much for your help. Best wishes, Linda

[moxtoby]

Hi Linda,

I have decided to use HDDM for my research, as I failed to get HSSM to work on Go/No-go data. If you do manage to get Go/No-go to work on HSSM, please do share the code!

Wishing you all the best in your study. Michelle

[LindaFiorini]

Hi Michelle,

First of all, thank you for answering! I have been trying to use HSSM for go no go for a month, I am giving up :) I could manage to use the docker. Still, I have some doubts on how to fit the model. Could you share the code? Thank you very much, Linda

On Tue, Dec 19, 2023 at 10:32 AM Michelle Oxtoby @.***> wrote:

Hi Linda,

I have decided to use HDDM for my research, as I failed to get HSSM to work on Go/No-go data. If you do manage to get Go/No-go to work on HSSM, please do share the code!

Wishing you all the best in your study. Michelle

— Reply to this email directly, view it on GitHub https://github.com/lnccbrown/HSSM/issues/173#issuecomment-1862414792, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBWHZ4756NRGUSOFQGM4CZTYKFNJZAVCNFSM6AAAAAA2AVBMTCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRSGQYTINZZGI . You are receiving this because you commented.Message ID: @.***>

[digicosmos86]

Hi @LindaFiorini and @moxtoby,

Unfortunately, HSSM does not support go no-go data yet. It will be the one of the first items on our agenda for the new year. Please stay tuned!

Thank you! Paul

[moxtoby]

Thank you @digicosmos86 Paul. I will keep my eye out for the changes in the new year.

Best wishes, Michelle

[pauldhami]

Greetings!

Thank you for the toolbox. I am hoping to soon dive into DDM specifically for some Go/No-Go data I have. I was just curious if there was a rough estimate as to when HSSM would support Go/No-Go data? I only ask since I was debating whether to potentially wait for it in the HSSM or start off with the HDDM.

Thank you.

[digicosmos86]

Hi @pauldhami ,

The latest dev version already supports go/no-go data. We just need some robust neural networks for that. Maybe @AlexanderFengler has an estimate? I am also happy to provide instruction on how to get the dev version up and running with the networks that we currently have, if you are interested in helping us out test the go/no-go support.

Thanks! Paul

[pauldhami]

Hi @pauldhami ,

The latest dev version already supports go/no-go data. We just need some robust neural networks for that. Maybe @AlexanderFengler has an estimate? I am also happy to provide instruction on how to get the dev version up and running with the networks that we currently have, if you are interested in helping us out test the go/no-go support.

Thanks! Paul

Hi Paul,

Ah my apologizes, in that case, instructions on how to get the dev version up and running would be great! Would love to start off with the HSSM right from the beginning of my project.

Thank you in advance, Paul

[digicosmos86]

Hi @pauldhami , The latest dev version already supports go/no-go data. We just need some robust neural networks for that. Maybe @AlexanderFengler has an estimate? I am also happy to provide instruction on how to get the dev version up and running with the networks that we currently have, if you are interested in helping us out test the go/no-go support. Thanks! Paul

Hi Paul,

Ah my apologizes, in that case, instructions on how to get the dev version up and running would be great! Would love to start off with the HSSM right from the beginning of my project.

Thank you in advance, Paul

Hi @pauldhami,

You just need to install HSSM through git instead of PyPI to set up the dev version. If you already have a virtual environment set up, you can simply update HSSM with this command:

pip install git+https://github.com/lnccbrown/HSSM.git

If you want to set up a new virtual environment and install the dev version, just follow our installation guide. At the last step, instead of installing HSSM using pip install hssm, use the command above.

To add support for go/no-go data, first, you need to download a network from here. We also have networks that support deadlines, and if that's the case, please let us know. Then, when you are creating the hssm model, simply use these extra arguments:

model = hssm.HSSM(
    data=...,
    ...,
    missing_data=True,
    loglik_missing_data="path_to_the_onnx_file_you_have_downloaded",
)

This assumes that the missing values in your data are coded as -999. Note that this network is not the final version that we are releasing. We are providing it just for testing purposes.

Let me know if you have any questions.

Thanks! Paul