search

SIMPLE-AstroDB / SIMPLE-db

Curated collection of data for low mass stars, brown dwarfs, and directly imaged exoplanets.
https://simple-bd-archive.org/
BSD 3-Clause "New" or "Revised" License
11 stars 22 forks source link

Ingest non-2MASS JHK photometry from Ultracool sheet #526

Open kelle opened 4 months ago

kelle commented 4 months ago

Part of #452

@kilohoku - I see some JHK measurements in the "2MASS" columns which don't appear to be 2MASS sources. Are the majority of these from a particular instrument/telescope that is not 2MASS?

kilohoku commented 4 months ago

For these non-2MASS sources, there's no dominant origin for the photometry.

A good chunk of it is synthetic or converted from some other system (e.g., by polynomial).

On Mon, Jul 1, 2024 at 1:36 PM Kelle Cruz @.***> wrote:

Part of #452 https://github.com/SIMPLE-AstroDB/SIMPLE-db/issues/452

@kilohoku https://github.com/kilohoku - I see some JHK measurements in the "2MASS" columns which don't appear to be 2MASS sources. Are the majority of these from a particular instrument/telescope that is not 2MASS?

— Reply to this email directly, view it on GitHub https://github.com/SIMPLE-AstroDB/SIMPLE-db/issues/526, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGONKVQRF4JY6CBJND3YHG3ZKGHQXAVCNFSM6AAAAABKGAE2DOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DIMZZGM2TMNI . You are receiving this because you were mentioned.Message ID: @.***>

-- Will Best Research Associate and Lecturer The University of Texas at Austin Department of Astronomy 2515 Speedway, Stop C1400 Austin, Texas 78712-1205

kelle commented 4 months ago

Given that we don't ingest synthetic or calculated photometry, i'm taking this action item off the list. Thanks for the quick response!

kilohoku commented 4 months ago

Makes sense.

One easy exception you could make: Everything in the Ks_2MASS columns with a reference of McMa13 or McMa21 is from VISTA, which has a very similar Ks filter to 2MASS.

On Mon, Jul 8, 2024 at 4:19 PM Kelle Cruz @.***> wrote:

Given that we don't ingest synthetic or calculated photometry, i'm taking this action item off the list. Thanks for the quick response!

— Reply to this email directly, view it on GitHub https://github.com/SIMPLE-AstroDB/SIMPLE-db/issues/526#issuecomment-2215190652, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGONKVQAFXNA3MBMBMBYE6DZLLXZTAVCNFSM6AAAAABKGAE2DOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJVGE4TANRVGI . You are receiving this because you were mentioned.Message ID: @.***>

-- Will Best Research Associate and Lecturer The University of Texas at Austin Department of Astronomy 2515 Speedway, Stop C1400 Austin, Texas 78712-1205

Exu-112 commented 3 months ago

It seems like the update to astrodb_utils a few days ago broke the find_source_in_db function. Im getting this error:

Traceback (most recent call last): File "/Users/florence/Astro/SIMPLE-db/scripts/ingests/ultracool_sheet/Ingest_VISTA_K_photometry.py", line 38, in <module> db = load_astrodb( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodb_utils/utils.py", line 95, in load_astrodb db.load_database( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodbkit2/astrodb.py", line 976, in load_database self.load_json(os.path.join(directory_of_sources, file)) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodbkit2/astrodb.py", line 909, in load_json conn.execute(self.metadata.tables[self._primary_table].insert().values(data[self._primary_table])) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1414, in execute return meth( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 485, in _execute_on_connection return connection._execute_clauseelement( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1630, in _execute_clauseelement compiled_sql, extracted_params, cache_hit = elem._compile_w_cache( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 664, in _compile_w_cache compiled_sql = self._compiler( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 290, in _compiler return dialect.statement_compiler(dialect, self, **kw) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 1269, in __init__ Compiled.__init__(self, dialect, statement, **kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 710, in __init__ self.string = self.process(self.statement, **compile_kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 755, in process return obj._compiler_dispatch(self, **kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 143, in _compiler_dispatch return meth(self, **kw) # type: ignore # noqa: E501 File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5317, in visit_insert crud_params_struct = crud._get_crud_params( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/crud.py", line 326, in _get_crud_params raise exc.CompileError( sqlalchemy.exc.CompileError: Unconsumed column names: dec, ra

It seems to relate to the change from ra and dec to ra_deg and dec_deg in the column names. I tried changing schema to reflect this, but I couldn't load the database then as the column names were different. The data files would need to be updated somehow if this change to schema is to be implemented.

Exu-112 commented 3 months ago

Running pytest reveals nothing wrong. Perhaps more tests are needed to check integration with astrodb_utils?

kelle commented 3 months ago

It looks like this error is actually in load_astrodb not find_source_in_db. One thing that has changed since you last ran this script is the change in the way the JSON files are organized. I tagged you in a comment on Slack that could help.