SIMPLE-AstroDB / SIMPLE-db

BSD 3-Clause "New" or "Revised" License
11 stars 22 forks source link

Bard14 Spectra #439

Closed LishaRamon closed 9 months ago

LishaRamon commented 9 months ago

Will be ingesting converted spectra from bard14.

Info from issue: "Spectra (17) converted in https://github.com/SIMPLE-AstroDB/SIMPLE-db/pull/365. Now we need to upload and ingest. bard14.zip

Filenames have spaces in it. Not sure if that will be thing or not.

Spectra uploaded to: https://bdnyc.s3.amazonaws.com/SpeX/Prism/ AWS converted spaces to plus signs ('+') in the filenames."

Link to relevant issue: Closes #366

For data ingests:

LishaRamon commented 9 months ago

https://docs.astropy.org/en/stable/io/fits/index.html

Not sure how to read in a folder into data for python to read. Looking up ideas of converting fits and png files into csv data then ingesting them (via loop)

kelle commented 9 months ago

You can use the pre-existing google sheet. Check out lines 8-13 in this script: scripts/spectra_convert/convert_bard14_spectra.py

LishaRamon commented 9 months ago

Is this the link to the sheet you mean? "https://bdnyc.s3.amazonaws.com/SpeX/Prism/" It's broken/non existent to me. Update: it's in the convert spectra.py file, nevermind got it

kelle commented 9 months ago

Instead of using the ingest_spectra function, I think we want to do use the update method of astrodbkit2 to update the fields. https://astrodbkit2.readthedocs.io/en/latest/#updating-data

Do this row-by-row inside the loop.

kelle commented 9 months ago

Once you get the loop working, I also want to check that the spectra links are valid. Here's some code to get you started.

import requests
request_response = requests.head(spectra_value)
            status_code = (
                request_response.status_code
            )  # The website is up if the status code is 200