The functions that download the lightcurves (i.e. cone_search) are crashing if run from inside multiprocessing environment, i.e. process pool. Originally encountered it while trying to use the library from inside Celery, but it also manifests in simple examples like that:
from pyasassn.client import SkyPatrolClient
import multiprocessing
def fn(x):
client = SkyPatrolClient()
lcs = client.cone_search(ra_deg=296.02191444, dec_deg=23.44665476, radius=10/3600, catalog='master_list', download=True, threads=1)
with multiprocessing.Pool(processes=1) as pool:
pool.map(fn, [1])
Error message is AssertionError: daemonic processes are not allowed to have children, coming from inside _get_curves(), and the reason is what there another multiprocessing.Pool is being created, even if threads=1 is specified. And that is not seemingly supported by multiprocessing module.
It would be extremely nice if a separate code path, not involving multiprocessing, was used there when threads=1.
Hi,
The functions that download the lightcurves (i.e. cone_search) are crashing if run from inside multiprocessing environment, i.e. process pool. Originally encountered it while trying to use the library from inside Celery, but it also manifests in simple examples like that:
Error message is
AssertionError: daemonic processes are not allowed to have children
, coming from inside_get_curves()
, and the reason is what there anothermultiprocessing.Pool
is being created, even ifthreads=1
is specified. And that is not seemingly supported by multiprocessing module.It would be extremely nice if a separate code path, not involving multiprocessing, was used there when
threads=1
.Thanks!