CosmoStat / autometacal

Metacalibration and shape measurement by automatic differentiation
MIT License
4 stars 1 forks source link

Add a baseline CFIS dataset #40

Closed EiffL closed 2 years ago

EiffL commented 2 years ago

This PR adds a first pass of a tf dataset of CFIS-like images, this aims at solving #39.

This is a first pass because we will probably want to add additional variants on that dataset in the future, but it is a starting point

EiffL commented 2 years ago

Super easy to use:

import autometacal
import tensorflow_dataset as tfds
import tensorflow as tf

# We create a function to add noise on the fly for data augmentation
@tf.function
def add_noise(example):
    im_noise = example['obs'] + example['noise_std'] * tf.random.normal([51,51])
    return im_noise, example

dset = tfds.load('CFIS/parametric_shear_1k', split='train')
dset = dset.map(add_noise)

for im, example in dset:
  # im contains the noisy observation
  # example contains psf image, observation without noise, and galaxy magnitude
  ....
aguinot commented 2 years ago

I checked the notebook, I am a bit confused by the results:

EiffL commented 2 years ago

Dont look too closely at the notebook. The response computed by autometacal is wrong, cf #26. The tqdm looks slower but that's because I do like 10000 instead of 1000 galaxies.

The main point of this PR is to add the dataset, the notebook is to illustrate how to load and use the dataset, but it's optional, if you think it's not useful I can drop it for now.