search

askap-vast / vast-pipeline

This repository holds the code of the Radio Transient detection pipeline for the VAST project.
https://vast-survey.org/vast-pipeline/
MIT License
7 stars 3 forks source link

Error when adding new images to run #718

Open ddobie opened 1 week ago

ddobie commented 1 week ago

Run is https://dev.pipeline.vast-survey.org/piperuns/77/, with log file 2024-06-26-05-36-59_log.txt

Traceback is

2024-06-26 07:01:03,006 runpipeline ERROR Processing error:
No objects to concatenate
Traceback (most recent call last):
  File "/usr/src/vast-pipeline/vast-pipeline-add-further-debug-logging/vast_pipeline/management/commands/runpipeline.py", line 340, in run_pipe
    pipeline.process_pipeline(p_run)
  File "/usr/src/vast-pipeline/vast-pipeline-add-further-debug-logging/vast_pipeline/pipeline/main.py", line 196, in process_pipeline
    sources_df = parallel_association(
  File "/usr/src/vast-pipeline/vast-pipeline-add-further-debug-logging/vast_pipeline/pipeline/association.py", line 1604, in parallel_association
    dd.from_pandas(images_df.set_index('skyreg_group'), npartitions=npartitions)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/base.py", line 292, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/base.py", line 575, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/multiprocessing.py", line 220, in get
    result = get_async(
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/local.py", line 508, in get_async
    raise_exception(exc, tb)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/local.py", line 316, in reraise
    raise exc
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/local.py", line 221, in execute_task
    result = _execute_task(task, data)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/core.py", line 119, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/optimization.py", line 990, in __call__
    return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/core.py", line 149, in get
    result = _execute_task(task, cache)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/core.py", line 119, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/utils.py", line 39, in apply
    return func(*args, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/dataframe/core.py", line 6350, in apply_and_enforce
    df = func(*args, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/dask/dataframe/groupby.py", line 170, in _groupby_slice_apply
    return g.apply(func, *args, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1414, in apply
    result = self._python_apply_general(f, self._selected_obj)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1455, in _python_apply_general
    values, mutated = self.grouper.apply(f, data, self.axis)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/groupby/ops.py", line 761, in apply
    res = f(group)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1388, in f
    return func(g, *args, **kwargs)
  File "/usr/src/vast-pipeline/vast-pipeline-add-further-debug-logging/vast_pipeline/pipeline/association.py", line 1143, in association
    sources_df, skyc1_srcs = reconstruct_associtaion_dfs(
  File "/usr/src/vast-pipeline/vast-pipeline-add-further-debug-logging/vast_pipeline/pipeline/utils.py", line 1549, in reconstruct_associtaion_dfs
    measurements = pd.concat(
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 347, in concat
    op = _Concatenator(
  File "/usr/src/vast-pipeline/.venv/vast-pipeline-B4hVHlAK-py3.8/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 404, in __init__
    raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate