Implement a CNN-based embedder for images and spectra - Githubissues

PolymathicAI / AstroCLIP

Multimodal contrastive pretraining for astronomical data

MIT License

77 stars 12 forks source link

Implement a CNN-based embedder for images and spectra #5

Closed EiffL closed 1 year ago

EiffL commented 1 year ago

The goal here is to establish a baseline using the state of the art so far that has been produced in astro for similar tasks.

Specifically:

Spender (Melchior and friends 2022) https://arxiv.org/abs/2211.07890 for spectra
Stein et friends 2021 https://arxiv.org/abs/2110.13151 https://github.com/georgestein/ssl-legacysurvey for images

The first architecture is a rather simple 1D CNN model, the second one is based on a standard resnet 50.

I'm starting to prototype this setup on this notebook here: https://github.com/FoundationModelsForScience/AstroCLIP/blob/main/notebooks/dev/ConvolutionalPrototyping.ipynb

EiffL commented 1 year ago

Ok, this is done, demo notebook is here: https://github.com/FoundationModelsForScience/AstroCLIP/blob/main/notebooks/dev/LightningTraining.ipynb

Now it's just a matter of copy/pasting the code into a nice script