search

bruvellu / cifonauta

Marine biology image database by CEBIMar/USP
http://cifonauta.cebimar.usp.br
GNU General Public License v3.0
21 stars 5 forks source link

Use `taxa.py` functions to handle taxonomic information instead of custom logic #297

Open bruvellu opened 4 months ago

bruvellu commented 4 months ago

The logic for querying, adding, and handling taxonomic information from WoRMS exists since March 2023 via update_taxa.py management command. I adapted this code into a general library named taxa.py with the goal of making these functions reusable for the new dashboard. However, the current taxonomic handling is running a custom logic using the low-level worms.py functions (see #296). There are differences in how the ancestor tree is fetched and how valid taxa are identified. In addition, the media from invalid taxa are not assigned to the correspondent valid taxa. This code needs to be revised and refactored to use taxa.py. I haven't tested extensively, but I'll paste some bugs I found below.

bruvellu commented 4 months ago

IndexError: list index out of range when no records are found:

[INFO] 2024-03-09 18:05:12,683 @ worms __init__ (l66): Initiating contact with WoRMS...
[INFO] 2024-03-09 18:05:13,146 @ worms __init__ (l69): Connected to WoRMS web services.
[INFO] 2024-03-09 18:05:13,146 @ worms get_aphia_records (l94): Searching for the name "Danio Rerio"
[INFO] 2024-03-09 18:05:13,395 @ worms show_results (l203): Found no records.
Internal Server Error: /api/taxa/
Traceback (most recent call last):
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/decorators.py", line 50, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/meta/views.py", line 69, in create_taxa
    taxon.update_taxonomic_tree()
  File "/home/nelas/Code/cifonauta/meta/models.py", line 684, in update_taxonomic_tree
    records = aphia.get_aphia_records(taxon_name)[0]
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
bruvellu commented 4 months ago

IndexError: list index out of range when no records for parent taxa are found:

[INFO] 2024-03-09 18:07:51,886 @ worms __init__ (l66): Initiating contact with WoRMS...
[INFO] 2024-03-09 18:07:51,935 @ worms __init__ (l69): Connected to WoRMS web services.
[INFO] 2024-03-09 18:07:51,935 @ worms get_aphia_records (l94): Searching for the name "Danio"
[INFO] 2024-03-09 18:07:52,302 @ worms show_results (l199): Found 4 record(s)
[INFO] 2024-03-09 18:07:52,302 @ worms print_record (l208): 314249 / Danio menoni / Barman, 1986 / Species / unaccepted
[INFO] 2024-03-09 18:07:52,302 @ worms print_record (l208): 268950 / Danionella / Roberts, 1986 / Genus / accepted
[INFO] 2024-03-09 18:07:52,302 @ worms print_record (l208): 1517455 / Danionidae / Bleeker, 1863 / Family / accepted
[INFO] 2024-03-09 18:07:52,302 @ worms print_record (l208): 267074 / Danioninae / Bleeker, 1863 / Subfamily / accepted
[INFO] 2024-03-09 18:07:52,325 @ worms get_aphia_records (l94): Searching for the name "Cypriniformes [id=3495]"
[INFO] 2024-03-09 18:07:52,504 @ worms show_results (l203): Found no records.
Internal Server Error: /api/taxa/
Traceback (most recent call last):
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/virtual/lib/python3.11/site-packages/rest_framework/decorators.py", line 50, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelas/Code/cifonauta/meta/views.py", line 69, in create_taxa
    taxon.update_taxonomic_tree()
  File "/home/nelas/Code/cifonauta/meta/models.py", line 702, in update_taxonomic_tree
    record_parent_taxon = aphia.get_aphia_records(parent_taxon)[0]
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
bruvellu commented 4 months ago

Other problems: