search

UHaifa-IS / whgazetteer-mehdie

World Historical Gazetteer - MEHDIE version
http://whgazetteer.org
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

matching bug #112

Closed sinairusinek closed 1 year ago

sinairusinek commented 1 year ago

Happening when trying to do a linking job (Damast2 to AlThurayya2).

Request Method: POST Request URL: https://tool.mehdie.org/datasets/223/recon/

Django Version: 2.2.28 Python Version: 3.9.2 Installed Applications: ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.gis', 'django.contrib.messages', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.staticfiles', 'allauth', 'allauth.account', 'allauth.socialaccount', 'bootstrap_modal_forms', 'captcha', 'celery_progress', 'django_celery_results', 'django_extensions', 'django_filters', 'django_resized', 'django_tables2', 'djgeojson', 'fontawesome', 'guardian', 'leaflet', 'mathfilters', 'multiselectfield', 'rest_framework', 'rest_framework.authtoken', 'rest_framework_datatables', 'rest_framework_gis', 'tinymce', 'accounts.apps.AccountsConfig', 'api.apps.ApiConfig', 'areas.apps.AreasConfig', 'collection.apps.CollectionConfig', 'datasets.apps.DatasetsConfig', 'elastic.apps.ElasticConfig', 'main.apps.MainConfig', 'places.apps.PlacesConfig', 'remote.apps.RemoteConfig', 'resources.apps.ResourcesConfig', 'search.apps.SearchConfig', 'traces.apps.TracesConfig'] Installed Middleware: ['django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.security.SecurityMiddleware']

Traceback:

File "/usr/local/lib/python3.9/dist-packages/django/core/handlers/exception.py" in inner

  1. response = get_response(request)

File "/usr/local/lib/python3.9/dist-packages/django/core/handlers/base.py" in _get_response

  1. response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.9/dist-packages/django/core/handlers/base.py" in _get_response

  1. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python3.9/dist-packages/sentry_sdk/integrations/django/views.py" in sentry_wrapped_callback

  1. return callback(request, *args, **kwargs)

File "/home/macbookpro/whgazetteer-mehdie/datasets/views.py" in ds_recon

  1. d2 = DatasetFile.objects.get(dataset_id=p_dataset)

File "/usr/local/lib/python3.9/dist-packages/django/db/models/manager.py" in manager_method

  1. return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/usr/local/lib/python3.9/dist-packages/django/db/models/query.py" in get

  1. raise self.model.DoesNotExist(

Exception Type: DoesNotExist at /datasets/223/recon/ Exception Value: DatasetFile matching query does not exist.

tomersagi commented 1 year ago

This problem is coming from the fact that the Althurayya dataset was not uploaded in TSV format. The matching system knows how to match tsv to tsv, it has no code to find out if the tsv exists and if not, process JSON format, convert it to TSV and then do the match.

tomersagi commented 1 year ago

posting to mehdi-er-snlwejaxvq-ez.a.run.app/uploadfile/ with files: /tmp/tmp8hshftfh/damast_20221104_134452_cyemq38.csv and /tmp/tmp8hshftfh/AnonymousUser_althurayya_20230502_182100.csv

For some reason the matching works, but the views.py code designed to parse the match result cant find the file in the DatasetsFile table because it is looking for a tsv file.

tomersagi commented 1 year ago

ok, the problem is that the DatasetFile ID does not match the ids in the Dataset table for some odd reason. Not the JSON at all. @sinairusinek try again