USGS-WiM / whispersservices

Django services for WHISPers
2 stars 1 forks source link

error message when save event leads to duplicates when user hits "save event" again #551

Closed JChipault closed 2 years ago

JChipault commented 2 years ago

Describe the bug I noticed that we had some duplicate events so I wrote the partner who created them to see if they were indeed duplicates to figure out if I should delete two events and this is what I got in reply:

"As for the events 202328-202330. I suspect this is one of the events I had trouble entering. When I attempted to Save Event for the case, I got an error message. I don’t recall exactly what the message was, only that it was nonspecific and instead of progressing to a new event page, remained on the same event page, leading me to believe the event wasn’t created. So I visually checked all the fields and then attempted to Save again, but received the same nonspecific message and remained on the same page, thus leading me to click on Save Event a third time.

This happened again with event # 202349. 202350, and 202351, but I realized what was happening (that is, creating duplicate cases without realizing it) I changed the details (species and locations) of the 2 duplicates event to match 2 additional cases."

I (jenny) set 202329 and 202330 as private while we investigate this bug, then will delete them.

To Reproduce I entered the same information as 202328 into the test site (Form data JSON below) and it saved fine (test site event: 171084). There were different event org text options in the test site than the live site, but the org IDs are the same so that shouldn't matter.

Also noting that 202349 (the first event in the second string of events that caused the partner problems) also occurred in South Carolina. Is it something about this county - Charleston, South Carolina - that is causing the error?

I did a search for recent events in that county to see if there were other instances of repeat events recently but it appears not. At a glance, 202373, 202376, and 202377 look like they could be duplicates but they all have different event reference numbers, which is what this partner uses to distinguish cases, so these are not repeats.

Form Status: VALID

{ "event_reference": "W22-367 A-C", "event_type": "1", "complete": false, "public": true, "staff": null, "event_status": "1", "legal_status": 1, "legal_number": "", "new_organizations": [ { "org": 914 }, { "org": 1056 }, { "org": 900 } ], "new_service_request": { "request_type": 0, "new_comments": [] }, "new_event_diagnoses": [], "new_comments": [], "new_eventgroups": [], "new_event_locations": [ { "name": "", "start_date": "2022-05-01T05:00:00.000Z", "end_date": "2022-05-10T05:00:00.000Z", "country": "30", "administrative_level_one": 53, "administrative_level_two": 2322, "latitude": null, "longitude": null, "land_ownership": "6", "gnis_name": "", "gnis_id": "", "site_description": "", "history": "Dozens of black vultures (approximately 100) in various stages of decomposition were fond dead near a black vultures roost around a retention pond ", "environmental_factors": "", "clinical_signs": "", "comment": "", "event_type": 1, "new_location_species": [ { "species": 721, "population_count": null, "sick_count": null, "dead_count": 3, "sick_count_estimated": null, "dead_count_estimated": 100, "priority": null, "captive": false, "age_bias": null, "sex_bias": null, "new_species_diagnoses": [] } ], "new_location_contacts": [] } ], "new_read_collaborators": [], "new_write_collaborators": [] }

aaronstephenson commented 2 years ago

Hi @JChipault, here's where I am with this right now...

I've tried to recreate this scenario several times, both submitting the pure JSON to the API and also just using the web client, and I every attempt has succeeded without errors.

I looked into the logs and did find an internal server error for the exact time of the creation of Event 202328 (note, however, that there were no other errors in any of the logs near the times that any of the other events were created by this user):

 Internal Server Error: /api/events/
 Traceback (most recent call last):
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
     response = get_response(request)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
     response = self.process_exception_by_middleware(e, request)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
     response = wrapped_callback(request, *callback_args, **callback_kwargs)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
     return view_func(*args, **kwargs)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/viewsets.py", line 125, in view
     return self.dispatch(request, *args, **kwargs)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/views.py", line 509, in dispatch
     response = self.handle_exception(exc)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/views.py", line 469, in handle_exception
     self.raise_uncaught_exception(exc)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
     raise exc
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/views.py", line 506, in dispatch
     response = handler(request, *args, **kwargs)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/mixins.py", line 19, in create
     self.perform_create(serializer)
   File "/www/htdocs/prod/whispersservices/whispersapi/views.py", line 193, in perform_create
     serializer.save(created_by=self.request.user, modified_by=self.request.user)
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/serializers.py", line 205, in save
     self.instance = self.create(validated_data)
   File "/www/htdocs/prod/whispersservices/whispersapi/serializers.py", line 1136, in create
     item.save()
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/rest_framework/serializers.py", line 205, in save
     self.instance = self.create(validated_data)
   File "/www/htdocs/prod/whispersservices/whispersapi/serializers.py", line 2489, in create
     admin_l2.name, admin_l1.name, admin_l1.abbreviation, country.abbreviation)
   File "/www/htdocs/prod/whispersservices/whispersapi/serializers.py", line 2033, in search_geonames_adm2
     grj = gr.json()
   File "/www/miniconda3/envs/prod/lib/python3.6/site-packages/requests/models.py", line 900, in json
     return complexjson.loads(self.text, **kwargs)
   File "/www/miniconda3/envs/prod/lib/python3.6/json/__init__.py", line 354, in loads
     return _default_decoder.decode(s)
   File "/www/miniconda3/envs/prod/lib/python3.6/json/decoder.py", line 339, in decode
     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
   File "/www/miniconda3/envs/prod/lib/python3.6/json/decoder.py", line 357, in raw_decode
     raise JSONDecodeError("Expecting value", s, err.value) from None
 json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It appears that the response from the Geonames service could not be parsed as JSON. The confusing thing to me is that that Geonames API call is wrapped in a Try-Except block, and if the Try fails the Except code should send an email to the WHISPers Admins saying that the request failed, along with error information, and then continue on with the Event creation process without the Geonames data (because it isn't strictly necessary). Did you happen to get an email at or around 7:51 AM on June 15? If not, I don't have a way to further debug this.

What I can do right now is also wrap the JSON parsing line in a Try-Except block, in case the current Try block is 'too large' and not working as expected.

Other than that, I can't really debug this any further without more information or the ability to recreate the error. I'm not even certain that this is the cause of the problem that the user reported, because I can't reproduce their situation and get an error that I could debug, and because I don't see any other errors in the logs near the times the other events were created by this user. It would be an enormous help if the user could tell us what the error message actually said, or even the gist of it, but with the information we have we're at a dead end.

I would recommend that you ask this user to please screen capture any errors they encounter and tell us what they were trying to do and what happened.

aaronstephenson commented 2 years ago

I changed the way the Try-Except blocks were built. I have a small suspicision that I was using a too-narrow Exception for the larger code block, so I replaced those with generic Exceptions and wrapped the single json parsing lines in the narrower RequestException. Implemented in 7838189, to be included in the next Test release.

JChipault commented 2 years ago

Ok, I can't repeat it either. I'll ask this user to let us know if it happens again. Thanks for investigating.