Doing source finding before cropping the training area

[eahussein]

Why do we have to crop the image first and then perform source finding separately on the training and the whole image?

Can't we just perform source finding on the whole image and then split the data into training and testing??

[eahussein]

there is a lot to this,

• it seems that they are selecting every 2 raws in the training DF, but why if we cropped an area to training why skip, or why to divide the data in 1/2
• it seems that they are using scorer.run() for comparing against the GT data, but no idea what the parameters mean, and what the training has to do with anything here.
• the training parameter
  • if it is true, means only get training area,
  • exclude training area

[eahussein]

The above method uses coordinates the following coordinates : 1400: {"ra_min": -0.2688, "ra_max": 0.0, "dec_min": -29.9400, "dec_max": -29.7265},

which is the same as the one used when cropping the data

so it seems that there is no data leakage

[eahussein]

New Questions.,

• why then there is still a duplication of sources
• why do we need to separate the training image from the full image, if we can just shuffle the whole df later from the full image
• why are we skipping every two rows ON THE TRAINING DATA

In the meantime, I will follow my own path unless someone convinces me otherwise

[eahussein]

That explains why when we sued ML on the testing set, there was no sign of duplications

[eahussein]

New Questions.,

• why then there is still a duplication of sources using the np.close
  • (maybe just the sources are close to each other (bad explanation, but the code above looks good) )
• why do we need to separate the training image from the full image, if we can just shuffle the whole df later from the full image
• why are we skipping every two rows ON THE TRAINING DATA -why are we using the smaller image in the training, and the bigger image in the testing, I don't understand.

In the meantime, I will follow their solution no problem

[eahussein]

training area before the slice:

training after the slice: