JPEG files not accepted in Records

[m-prodan]

Describe the bug in current situation User reporting that a common file type they deal with is .jpeg - currently the records app can accept .jpg file formats, but not .jpeg

This is a common file type for MSD personals and will need to be resolved

Link bug to the User Story

Impact of this bug High - specifically for MSD requests

Chance of Occurring (high/medium/low/very low) High - user noted requests regularly have files in jpeg format.

Pre Conditions: which Env, any pre-requesites or assumptions to execute steps?

Steps to Reproduce Steps to reproduce the behavior:

1. Go to Any request in call for records with a division added
2. Upload a .jpeg file
3. See error

Actual/ observed behaviour/ results

Expected behaviour A clear and concise description of what you expected to happen. Use the gherking language.

Screenshots/ Visual Reference/ Source If applicable, add screenshots to help explain your problem. You an use screengrab.

[milosdes]

Added ".jpeg" under "dedupe" attribute to following files: https://citz-foi-prod.objectstore.gov.bc.ca/records-upload-formats/recordsuploadformats.json dev https://citz-foi-prod.objectstore.gov.bc.ca/records-upload-formats/dev/recordsuploadformats.json test https://citz-foi-prod.objectstore.gov.bc.ca/records-upload-formats/test/recordsuploadformats.json prod https://citz-foi-prod.objectstore.gov.bc.ca/records-upload-formats/prod/recordsuploadformats.json removed until further testing done

[nimya-aot]

Jpeg files are not able to upload on records tab. It is not fixed

[abin-aot]

expecting code changes on Harms Stitching! and other areas(need to analyze) whereever there is hardcoding on file extensions. ideally, all hardcoding should come from a ENV VAR or configmap file- which is a config!!! Can be treated as a tech. debt.

[m-prodan]

Interesting enough, I did ask the prod user to try the jpegs again based on Milos' activity, they mentioned it did work - but I can see we rolled it back. I'm not sure if they went the next step and tried to download for harms so not sure about the stitching part. Either way, looks like we'll need some more work on this based on the comments

[nimya-aot]

@milosdes Still not able to upload JPEG files. Could you please check? Private Zenhub Image

[KyEggleston]

If this isn't working before end of day, we can pull it out of 8.4.5 release and move to 9.0 release. Let's not let this hold us back from doing the 8.4.5 release - confirmed with Matt ok. @milosdes @nimya-aot

[milosdes]

@KyEggleston noted, I think Abin also mentioned the possibility of doing an 8.4.6 release for the .jpeg fix. We can sort it out in standup.

Just to document the issue - this is working on dev, but not on test for some reason. Below are the logs from the pods on dev and on test, as well as a screenshot that shows the recordsuploadformats.json is updated and loading the correct file with the .jpeg extension in test.

A git diff comparing main and dev shows the codebase is the same (with the exception of sonarcloud.properties file). Not sure what else could be causing the issue in test.

logs for d7abee-test, request-management-api-289-srpj2 pod: [{'s3uripath': 'https://citz-foi-prod.objectstore.gov.bc.ca/edu-test-e/EDU-2022-66551/4c9f142a-9284-41bf-8f54-877ddb25ec87.jpeg', 'attributes': {'lastmodified': '2023-10-30T23:23:27.803Z', 'filesize': 348350, 'divisions': [{'divisionid': 2}], 'batch': '84e785b9-5caa-43b1-bba2-f72fb1b18212', 'extension': '.jpeg', 'incompatible': False}, 'filename': 'Sample_0.jpeg', 'recordid': 11273}]

[2023-10-31 05:19:26,080] ERROR (root) <recordservice.py:recordservice.py:268>.__bulkcreate: Doc Reviewer API was given an unsupported file type - no job triggered - Record ID: 11273 File Name: Sample_0.jpeg

logs for d7abee-dev, request-management-api-782-snjl5 pod: [{'attributes': {'lastmodified': '2023-10-30T23:23:27.803Z', 'filesize': 348350, 'divisions': [{'divisionid': 2}], 'batch': '9d9f9f3d-510a-4f5e-add4-08005502908a', 'extension': '.jpeg', 'incompatible': False}, 's3uripath': 'https://citz-foi-prod.objectstore.gov.bc.ca/edu-dev-e/ABC-2099-3567830/15f5bd2f-6e7f-43a3-8528-16765951d610.jpeg', 'filename': 'Sample_0.jpeg', 'recordid': 12608}]

Private Zenhub Image

[milosdes]

@KyEggleston @m-prodan this is working in test as well now

[KyEggleston]

@milosdes - In TEST, confirmed that JPEG files are being uploaded successfully now, and run through dedupe and conversion. Duplicate JPEGs are also flagged, similar to other accepted files. Tested JPEG attachments to emails down multiple levels, no issues. Confirmed JPEG files are also being included in the consolidated PDF Harms Package (and not being treated as incompatible files, where the native file is included as a separate file). They also appear in the Redaction App in Test-Marshal now too.

[KyEggleston]

@nimya-aot - feel free to double check with same test cases/files as you did previously - otherwise, if working, we can close and leave in 8.4.5 release if possible.

[nimya-aot]

@KyEggleston Its working for my files too. You can close that