wfau / gaiadmpsetup

Python library used to create the Spark SQL tables required by GaiaDMP
GNU General Public License v3.0
0 stars 3 forks source link

Importing eDR3 into DR3 needs '.' to locate the files #9

Closed Zarquan closed 1 year ago

Zarquan commented 2 years ago

Running v0.1.4 on a test deployment reveals a problem with the import statements.

Fail to execute line 2: import gaiadmpsetup
Traceback (most recent call last):
....
  File "/usr/local/lib/python3.7/site-packages/gaiadmpsetup/gaiadr3_pyspark_schema_structures.py", line 1571, in <module>
    from gaiaedr3_pyspark_schema_structures import tmasspscxsc_best_neighbour_schema, twomass_psc_schema, allwise_best_neighbour_schema, allwise_sc_schema, panstarrs1_best_neighbour_schema, panstarrs_dr1_otmo_schema
ModuleNotFoundError: No module named 'gaiaedr3_pyspark_schema_structures'

The top level gaiadmpsetup.py has the following import statements:

from . import gaiaedr3_pyspark_schema_structures as edr3
from . import gaiadr3_pyspark_schema_structures as dr3

Using '.' to load the modules from files in the same directory.

The DR2 schema has the following import statement:

from gaiaedr3_pyspark_schema_structures import tmasspscxsc_best_neighbour_schema, ....

which probably needs a '.' to load the eDR3 schema from a module file the same directory.

Alternatively, if the eDR3 schema structures have already been loaded into the main Python module as edr3, then it might be possible to access them by using the edr3 prefix in the DR3 schema.

Changing

    'gaia_source_tmasspsc_best_neighbours' : 
        ([tmasspscxsc_best_neighbour_schema, twomass_psc_schema], release_folder + '/GDR3_2MASSPSC_BEST_NEIGHBOURS'),

to

    'gaia_source_tmasspsc_best_neighbours' : 
        ([edr3.tmasspscxsc_best_neighbour_schema, edr3.twomass_psc_schema], release_folder + '/GDR3_2MASSPSC_BEST_NEIGHBOURS'),
NigelHambly commented 2 years ago

My bad again sorry - as discussed & agreed, assigned to @stvoutsin to fettle.

stvoutsin commented 2 years ago

Patch for this has been committed and merged, and I've recreated v0.1.4 so no need for changing the version on the main gaiadmp repo

Zarquan commented 1 year ago

Is this done ?

stvoutsin commented 1 year ago

Yep this is done