As an Administrator I want to be able to load AVV XML catalogues onto the host machine so that they can be read when the server restarts

[dptolle]

As an Administrator I want to be able to load AVV XML catalogues onto the host machine so that they can be read when the server restarts.

The upload should be available via the parse dashboard. Uploading should automatically convert the XML into the required json format. Failure to convert should delete the file and remove the entry fmor the db. Deletion via the dashboard should also delete the created json. Success should be logged as well as displayed as a popup. Failure should also be logged.

[dptolle]

@afalenski Does this ticket cover everything we want to implement in the first instance?

[dptolle]

@afalenski For a complete test a server restart is required, however you can start by testing upload and deletion. I have only tested it with a small file (avv303). 'catalogueFile' is a required field. All else the programm should read from the file if available. The file content acts as the ultimate source of truth and will overwrite administrator added version and catalogueCode. You should be able to override the validFrom. Is this the way you want it? Also see if a longer upload (of a larger file) causes the frontend to become unresponsive - it shouldn't

[afalenski]

• Upload via parse dashboard is possible; for 20kb files the new row in the dashboard is created immediately; for a 13 MB file it took 24 seconds until the row was created; for all files which have the filename structure ###_v#-##.xml (with ### as the catalogue number and v#-## as version number) the dashboard created automatically separate column entries for catalogue number, version number and 'valid from' date., which is cool.
• However, uploading a 37.7 MB XML file, the creation of the dashboard row took 1 minute (I could live with that) and the values for catalogue, version and valid from were empty (see screenshot below). According to the JSON filename "v8-03.xml" also the JSON creation failed.
• An automatic deletion of this row did not take place (or: when is this supposed to happen? After server restart?).
• The error log message "2024-02-16T15:36:10.591Z - Unable to create Catalogue from file." is there, so something is logged. However, the time is not correct (1h behind) and the text contains too little information to find a solution to the problem. Could it say "file is too large" or "version number not found"?

@dptolle In catalogue 324 the version is in row 672. Could this perhaps be the reason for the bug? In other catalogues the version is in row 13 or 15.

• Whether a JSON file is created cannot be checked because the link goes to nowhere. A JSON filename is shown.
• Selecting a row and clicking on "Edit/Delete this row" the row disappears. Whether the JSON is deleted, too, cannot be seen.
• During the test the log (both Info and Error) became very slow meaning that more than 30 seconds nothing changed when switching from Info to Error. Being in the Browser menu and clicking on Logs, the page is white for several seconds and after some time text is shown. After writing these lines here the Error log showed the message "No error logs in the last 30 days". To be able to learn from errors these should be recorded until the user deletes them; perhaps an auto-delete for entries older than 3 months for logs bigger than 1 MB could prevent the log from becoming too large.
• When the row was deleted, a popup appeared in the lower right corner. It showed up only very briefly and showed some cryptic character combination. It would be more helpful, if the popup appeared for 3 seconds and showed a message like "Catalogue 322 was deleted."

Overall opinion: It is a good start (see first bullet point). If errors happen I would like to be informed in more detail and I would like to be able to display the JSON file (in a browser tab or as download). Thus, if there is a strange reaction to a Einsendebogen test file I could check in the JSON file whether the entry is there and whether it is correct. This might rule out some reasons for the error and shorten the time for the developer to solve a bug ticket.

[dptolle]

@afalenski Please test again. You still won't be able to see the JSON file through the Dashboard interface - this is a bug caused by our unusual deployment configuration. It might be fixed in the future, once the parse server is fully operational. In the mean time there is a work-around I will show you next time we meet.

[afalenski]

• Uploading a small (AVV316) and a big file (AVV324) worked. For the 37 MB catalogue the upload was complete after 72 seconds.
• Catalogue code, version and validFrom are shown.
• Uploading the same catalogue again resulted in a nice message in a red info box in the bottom right corner.
• Deleting a catalogue needs to be confirmed (good!) and a white box with light blue frame informs about the successful deletion.
• Adding an XSD file results in the warning "unsupported file type", adding an FCL JSON file results in the warning "catalogue code is required" and adding a KNIME settings.xml results in the warning "cannot read properties of undefined (reading 'metadaten')".