search

astronomy-commons / axs

Astronomy eXtensions for Spark: Fast, Scalable, Analytics of Billion+ row catalogs
https://axs.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
23 stars 12 forks source link

ADQL support #21

Open Dav-v opened 4 years ago

Dav-v commented 4 years ago

Hi, are there any plans to add ADQL support to AXS? Since a lot of archives are already using ADQL as their query language it could be useful to have also this interface, not only for cross-matches but even for other database queries (cone searches etc.). I see that there are some libraries that could help translating an ADQL query into an SQL one, like ADQL Library (Java) and queryparser (python). Could this be feasible? The biggest problem that I could think of would be how to ensure that AXS's optimizations are used even when plain ADQL queries are launched, so that the user doesn't have to manually specify to join on cat1.zone = cat2.zone during a cross-match , or to exclude duplicates for example. A solution could be to add some new ad-hoc ADQL functions or (maybe better but probably more difficult) to add a sort of optimization layer during the ADQL->SQL translation that automatically takes these aspects into account.

Thanks, Davide Viero

mjuric commented 4 years ago

@ctslater has been working on something related over the past few months (a TAP service front-end). I think it looks doable, but some aspects of it are really messy (with how Spark implements Hive support). Colin can add more.

Dav-v commented 4 years ago

Ok thanks, of course a full TAP front-end would be even better