ThomasAlbin
commented
1 week ago Hey,
this look very promising! Thanks a lot for the hint and all the information. Indeed, CLIP is a model developed by OpenAI to "link text and images" and is also used as one core element of image generators like Stable Diffusion.
AstroCLIP is like ... "the CLIP of astronomy"; mapping high dimensional data to a latent space. So your understanding is correct.
I put this in my backlog. I will dive into this soon. Thank you!
A suggestion for a long-term project. I recently found this interesting paper AstroCLIP: A Cross-Modal Foundation Model for Galaxies.
As far as I understand, the project focuses on developing a machine learning model that processes different types of data (e.g., images, time series, spectral data) from astronomical observations. The model is trained to distinguish between similar and dissimilar data points by creating representations that pull similar data closer and push dissimilar data apart within a learned feature space.
By exposing the model to various forms of observational data (e.g., telescope images, spectroscopy, light curves), the pretraining process teaches the model to align related information across different modalities while distinguishing between distinct classes (e.g., galaxies, stars, asteroids). This is the Github link of their project AstroCLIP.
I’d love your input to explore similar work and engage with coding tutorials or simplified versions of AstroCLIP to learn about the latest developments and technologies.