Cone search slower than similar region query.

astronomy-commons / axs

Astronomy eXtensions for Spark: Fast, Scalable, Analytics of Billion+ row catalogs

https://axs.readthedocs.io/

BSD 3-Clause "New" or "Revised" License

23 stars 12 forks source link

Cone search slower than similar region query. #4

Open ctslater opened 5 years ago

ctslater commented 5 years ago

@ebellm reports that the cone search is much slower (~100x) than a region search for 10"-sized areas. I'm guessing this is due to the pandas_udf involved in the cone search; maybe this distance function can get moved into scala to speed up the search?

zecevicp commented 5 years ago

I'll look into this.

ctslater commented 3 years ago

I've been unable to recreate this, using either very big or very small regions. It's true that very small searches are not very efficient (they take similar time as much larger areas), but both cone and region seem to be similarly slow in that regime (in the case of ZTF select * as a worst case scenario, several minutes).