I want to get my hands on your data!

[HaihaoSHI]

I read your brilliant paper "A deep-learning algorithm to disentangle self-interacting dark matter and AGN feedback models" and was thoroughly impressed. Any chance you could share the allSimData_binning_20.pkl file and other datasets you referenced? I promise to handle them with the utmost care! Any extra data or instructions would be a bonus. Thanks a million!

[davidharvey1986]

Hi HaiHaoSHI. Sure. Here you go.

https://drive.google.com/file/d/1x-l2FiQHqqPFQdN3oNoxj5MWJ7P0ZejG/view?usp=share_link

let me know if this works.

[HaihaoSHI]

Okay, I've submitted my access request on Google Drive.

[davidharvey1986]

Try this instead https://drive.google.com/file/d/1bYoNGAvwMtUv70L_u-aXosm4vXks-D6u/view?usp=share_link

[HaihaoSHI]

ok, I have downloaded the dataset, thank you very much. By the way, I heard that some hackers can attack your account from your shared link, so posting the shared link in the public chat box may not be a good option.

[HaihaoSHI]

I'd like to ask for your advice on some issues we've encountered. Using the allSimData_binning_20.pkl data file to reproduce your results (binned to 20x20 and using the DarkInception network), we found the model tends to overfit. The validation accuracy fluctuates between 77-79, while training accuracy reaches 85 or higher, which doesn't match the results in your paper.

I noticed the notebooks in the repository use pickles/binned_data_20.pkl. Are there differences between these datasets? Is binned_data_20.pkl already processed to 20x20? If so, could you kindly share binned_data_20.pkl with me, or provide any suggestions to help solve the overfitting issue?

And my collaborator has sent you two emails with questions, which might have been marked as spam. If you haven't received our emails, please let me know, and we'll try to resolve the issue.

[davidharvey1986]

Hi HaihaoSHI

Indeed I used a slightly different data file, I can send that through, however, I will say I never used darkinception I used inceptionV4 throughout.

THanks David

On 20 Jun 2024, at 07:44, HaihaoSHI @.***> wrote:

I'd like to ask for your advice on some issues we've encountered. Using the allSimData_binning_20.pkl data file to reproduce your results (binned to 20x20 and using the DarkInception network), we found the model tends to overfit. The validation accuracy fluctuates between 77-79, while training accuracy reaches 85 or higher, which doesn't match the results in your paper.

I noticed the notebooks in the repository use pickles/binned_data_20.pkl. Are there differences between these datasets? Is binned_data_20.pkl already processed to 20x20? If so, could you kindly share binned_data_20.pkl with me, or provide any suggestions to help solve the overfitting issue?

And my collaborator has sent you two emails with questions, which might have been marked as spam. If you haven't received our emails, please let me know, and we'll try to resolve the issue.

— Reply to this email directly, view it on GitHub https://github.com/davidharvey1986/darkCNN/issues/1#issuecomment-2179862858, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWLQCCBP6WCLWKYQROOQHLZIJT33AVCNFSM6AAAAABIRDWZLSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZHA3DEOBVHA. You are receiving this because you commented.

[HaihaoSHI]

Yes, I really need the dataset used in your article. If you could send it to me, please send the download link or file to my email: shi_haihao@outlook.com. Thank you very much!

[davidharvey1986]

Can you please explain your project and purpose with the data? -------- Original message --------From: HaihaoSHI @.> Date: 20/06/2024 11:47 (GMT+01:00) To: davidharvey1986/darkCNN @.> Cc: davidharvey1986 @.>, Comment @.> Subject: Re: [davidharvey1986/darkCNN] I want to get my hands on your data! (Issue #1) Yes, I really need the dataset used in your article. If you could send it to me, please send the download link or file to my email: @.*** Thank you very much!

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[HaihaoSHI]

We want to try using some of the latest, more interpretable models, like "Kolmogorov–Arnold Networks," to uncover the hidden patterns behind classifications and achieve better AI for science. We think your work is impressive. First, we plan to use these new networks to replicate your results, and then we'll build on that. Achieving the same results as in your paper is a crucial step for us. If we succeed, we hope to use this project as our graduate thesis topic.

[davidharvey1986]

https://drive.google.com/file/d/1D0oX5S5qotAvs-I0wCCsROEel05YCGaX/view?usp=share_link

[HaihaoSHI]

Thank you very much!