Run MambaMIL multiple times, and each time the results were different on Camelyon16

[haiqinzhong]

@wyhsleep @isyangshu
Hello, did you test your model on Camelyon16? I ran it multiple times, and the results were different. When I run the setup.py to install mamba_ssm, the result of acc and AUC will be around 92% and 95%. But when I put it locally (just like the form you provided, put the folder directly in the project), the acc and AUC results were only around 87% and 93%. Of course, whether install it or just put it locally, the results between different run times, the results also different, even if the random_seed is the same. The code I'm using is the MambaMIL.py and mamba folders you uploaded the first time. The train code uses the TransMIL code. Looking forward to your reply.

[isyangshu]

For the first question, you mean that you attempt two ways to employ SRMamba?

• You directly install official mamba and transfer our SRMamba into the lib/python3.10/site-packages/mamba_ssm/?
• You use our provided folder and just run the setup.py? The folder we provided contains a specific version of Mamba, and we only insert our own SRMamba without changing anything else. Specifically, at present, the environment directly installs mamba, and call SRMamba in our provided folder. I guess the performance difference comes from the version of Mamba or causal-conv? We use ten-fold cross validation for Camelyon16 and get the result of acc 91.32% and auc 95.49%.

For the instability of Mamba, you can refer to https://github.com/state-spaces/mamba/issues/137. You can use a smaller learning rate, which may allow you to reach the same local optimum before the backpropagation divergence occurs when you run the code multiple times.

[haiqinzhong]

Thank you for your reply. I just use your provided folder and run the setup.py After that, I can use two ways to employ SRMamba.

• from mamba_ssm import SRMamba (which is located lib/python3.10/site-packages/mamba_ssm/)
• from mamba.mamba_ssm import SRMamba (which is located in the project, for example, TransMIL/mamba/mamba_ssm)

And for the problem of the instability of Mamba, thank you for your link. Now I know that when running the code multiple times, it can get different results is normal. The reason I got a lower result may be the split datasets are different.