department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
281 stars 197 forks source link

526 txt file uploads to are being rejected EVSS #37947

Open michelpmcdonald opened 2 years ago

michelpmcdonald commented 2 years ago

Issue Description

The website now allows users to upload supporting evidence text files when creating claim.

Evss is rejecting the text file, can view in sidekiq dead queue:

[{"key"=>"EVSS_15004", "text"=>"We were unable to complete your request because your file type is not a supported format. Try using a different file type such as PDF (unlocked), JPEG, BMP or TXT. If you still have problems, call the VA at 1-800-827-1000 (711, if you use TDD), Monday - Friday. 8:00 am - 9:00 pm ET. When you call, provide the following error code: EVSS_15004.", "severity"=>"ERROR"}]

EVSS suggested the 'Content-Type' header to text


Tasks

Acceptance Criteria

michelpmcdonald commented 2 years ago

Background: txt files users send can be in many different encodings.

EVSS only accepts ASCII encoded text files. ASCII is a basic encoding that allows for 128 characters.

In the instances that failed, so far I've found Windows-1252 and macRoman encoded files.

There is no 100% correct way to automatically determine a txt files encoding, but to properly convert a txt file to the ASCII encoding evss accepts, the encoding of the files needs to be known.

Going to try to make a guess if I can, otherwise provide a best effort conversion that may not be perfect, but it should be readable. For example "I'm" might turn into "I?m". In general, Letters and Numbers will translate clean...

michelpmcdonald commented 2 years ago

As of July 14th, 1850 attempted file uploads have hit this issue.