search

SamSweere / xmm-superres-denoise

Deep Learning-Based Super-Resolution and De-Noising for XMM-Newton Images.
7 stars 1 forks source link

About datasets posted in directory `train -> display_datasets -> xmm_sim_display_selection` #25

Open leungzzz opened 2 weeks ago

leungzzz commented 2 weeks ago

When the params log_images_every_n_epochs:5 and display -> sim_display_name: xmm_sim_display_selection be set, which kinds of dataset should be posted in directory train -> display_datasets -> xmm_sim_display_selection as described in xmm_display_datamodule.py?

At this time i have scannd carefully all of four branches, neither of them mention it. I really just confused aboult it, sorry to bother again!

bojobo commented 2 weeks ago

Hi! No worries, I'm happy that you're interested in the project.

I was sure, that I have uploaded it... To make matters worse, I can't access the cluster where the data is at and I don't know why. I'll upload the images to huggingface as soon as I have access again. This could take some time.

@SamSweere Do you still have the display_datasets? Maybe you could share it.

Otherwise @leungzzz you can randomly choose images from the simulated dataset with the chosen exposure (see here) and use those images. They are just there to have some visual feedback.

SamSweere commented 2 weeks ago

Hi @leungzzz and @bojobo,

It's great to see more people interested in working on ML in astronomy!

I have a local copy of all the data, which I can share. However, please note that the full dataset is approximately 200 GB. I've uploaded the display datasets and some smaller subsets that I used for development to Google Drive: Google Drive Link.

Please note that I will likely remove this shared folder in a few weeks. If you need the full dataset, we can arrange a peer-to-peer transfer.

bojobo commented 2 weeks ago

Thanks! I'll download display_datasets and push them to huggingface.

leungzzz commented 2 weeks ago

That's great to get your reply, @bojobo . At that time prepare to randomly choose images from the simulated dataset, then get that reply from you @SamSweere. Now i have got that xmm_selection_datasets from your temporarily shared links. Please let me know what preparations should I need to do if we build a peer-to-peer transfer, I just really have no much experience (not afraid to say).

It's really appreciate to get you two reply so timely.

bojobo commented 2 weeks ago

You're welcome!

If you have got xmm_sim_display_selection out of the compressed file available through Sam's link, then you're ready to go. The whole dataset about which Sam's talking is completly available at huggingface. There is no need to get the data from Sam.