django-daiquiri / daiquiri

A framework for the publication of scientific databases
https://escience.aip.de/daiquiri
Apache License 2.0
26 stars 8 forks source link

BUG: Error downloading table without ucd information #148

Closed glaubervila closed 1 year ago

glaubervila commented 1 year ago

hello, I'm testing daiquiri, and when trying to download a table in votable format, I got an error, the other formats worked correctly, looking at the log found that the problem was caused by the lack of ucds. Would it be possible to have this log message as feedback in the interface? or maybe disable the votable download function when the table doesn't meet the requirements.

I'm in the first contact with the daiquiri, maybe I'm doing something wrong, please forgive me if I missed something. Thanks in advance.

Steps to reproduce the error:

[2023-02-09 15:45:01,791] ERROR: Internal Server Error: /daiquiri/query/api/jobs/08bf4890-f225-4c88-a518-f15430651564/download/table/
Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/query/viewsets.py", line 286, in create_download
    download_job = download_job_model.objects.get(query_job=job, **params)
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py", line 496, in get
    raise self.model.DoesNotExist(
daiquiri.query.models.DownloadJob.DoesNotExist: DownloadJob matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/opt/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/opt/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/query/viewsets.py", line 308, in create_download
    download_job.run()
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/query/models.py", line 444, in run
    create_download_table_task.apply((download_id, ), task_id=download_id, throw=True)
  File "/opt/venv/lib/python3.9/site-packages/celery/app/task.py", line 793, in apply
    ret = tracer(task_id, args, kwargs, request)
  File "/opt/venv/lib/python3.9/site-packages/celery/app/trace.py", line 468, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/opt/venv/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/query/tasks.py", line 289, in create_download_table_task
    raise e
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/query/tasks.py", line 280, in create_download_table_task
    for line in download_job.query_job.stream(download_job.format_key):
  File "/opt/venv/lib/python3.9/site-packages/daiquiri/core/generators.py", line 97, in generate_votable
    if 'meta.id' in field['ucd'] and 'meta.ref' in field['ucd']:
TypeError: argument of type 'NoneType' is not iterable

VOTable used:

<?xml version="1.0"?>
<VOTABLE version="1.3"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://www.ivoa.net/xml/VOTable/v1.3"
    xmlns:stc="http://www.ivoa.net/xml/STC/v1.30">
    <RESOURCE type="results">
            <INFO name="QUERY_STATUS" value="OK" />
            <INFO name="QUERY" value="Select top 5 coadd_object_id, ra, dec from des_dr2.coadd_objects" />
            <INFO name="QUERY_LANGUAGE" value="adql-2.0" />
            <INFO name="SOURCE" value="des_dr2.coadd_objects" />
        <LINK title="des_dr2.coadd_objects" content-role="doc" href=""/>
        <TABLE name="daiquiri_user_gverde.2023-02-09-15-43-21-293798">
            <FIELD name="coadd_object_id" datatype="long" />
            <FIELD name="ra" datatype="double" />
            <FIELD name="dec" datatype="double" />
            <DATA>
                <TABLEDATA>
                    <TR>
                        <TD>954200778</TD>
                        <TD>333.112355</TD>
                        <TD>-61.026571</TD>
                    </TR>
                    <TR>
                        <TD>957705680</TD>
                        <TD>333.868251</TD>
                        <TD>-61.69803</TD>
                    </TR>
                    <TR>
                        <TD>954201199</TD>
                        <TD>332.806105</TD>
                        <TD>-61.022547</TD>
                    </TR>
                    <TR>
                        <TD>954213924</TD>
                        <TD>333.3871</TD>
                        <TD>-61.14554</TD>
                    </TR>
                    <TR>
                        <TD>954203046</TD>
                        <TD>333.280633</TD>
                        <TD>-61.042644</TD>
                    </TR>
                </TABLEDATA>
            </DATA>
        </TABLE>
    </RESOURCE>
    <RESOURCE type="meta" utype="adhoc:service">
        <PARAM name="standardID" datatype="char" arraysize="*" value="ivo://ivoa.net/std/DataLink#links-1.0" />
        <PARAM name="accessURL" datatype="char" arraysize="*" value="http://localhost/datalink/links" />
        <GROUP name="inputParams">
            <PARAM name="ID" datatype="char" arraysize="*" value="" ref="datalinkID"/>
        </GROUP>
    </RESOURCE>
</VOTABLE>
kimakan commented 1 year ago

Thank you for bringing up the issue! It is a bug and we will fix it shortly.

agy-why commented 1 year ago

@glaubervila issue was fixed with #149 Now merged in master . feel free to pull.

I wish you a nice day