Investigate error with activity file XML import to IATI Publisher

younginnovations / iatipublisher

IATI Publishing Tool

GNU Affero General Public License v3.0

7 stars 0 forks source link

Investigate error with activity file XML import to IATI Publisher #1463

Open emmajclegg opened 1 week ago

emmajclegg commented 1 week ago

This affects a single publisher - the details can be found in this Zendesk support ticket: https://iati.zendesk.com/agent/tickets/44547

PG-Momik commented 6 days ago

@emmajclegg A quick update on this,

We've looked into the the issues mentioned in the ticket, namely:

Files not being imported
Activity duplication

Files not being imported

Recent changes made in #1407 was missing a null check when resolving if any elements were using deprecated code. I've added the necessary null check. This issue has been fixed and is being tested.

Activity duplication

We've confirmed that some activities have indeed been duplicated. All of the duplicated activities were uploaded to the system via xml. We're looking into recreating the duplication issue. We haven't been able to pin point the exact cause for duplication itself. I'll keep you updated on the progress.

cc: @praweshsth @sarinasindurakar

PG-Momik commented 4 days ago

Hi @emmajclegg. Futher updates on this.

Files not being imported

The null check required to fix this issue has been deployed to production alongside #1462 . The issue mentioned in the Zendesk ticket will not occur not. Please let us know if the issue persists.

Activity duplication

We've found the cause of activity duplication when being updated via XML. The code to check if an activity being imported via XML already exists in database is failing. If a publisher tries to update/import an already existing activity via XML, but has added extra spaces in the <iati-identifier> field. EXAMPLE: <iati-identifier>IDENTIFER{extra space}</iati-identifier> The system considers this a new activity, resulting in the duplication.

We'll look into if any more publishers have similar issues and let you know of the possible solutions.

cc: @praweshsth @sarinasindurakar

emmajclegg commented 3 days ago

Thanks for the update @PG-Momik .

File not being imported

Thanks for applying the fix - I've tested the XML file upload on my test account and it looks to be working. I've let the publisher know.

Activity duplication

When you can, please send me a list of the duplicate activities (either by email or in this support ticket: https://iati.zendesk.com/agent/tickets/44548) and I'll work with the publisher to remove the duplicates. If there is an automated way that you can see if activities are the same apart from spaces in their identifier(?), please let me know as that would make a bulk deletion easier your side.

sarinasindurakar commented 5 hours ago

@PG-Momik

[ ] 1. When the user imports the file from an XML file and adds spaces before and after the identifier, the file is imported, but a random number is being set in the IATI identifier.

[ ] 2. Activities are still being duplicated when the user imports the file with added spaces.