SPIKE: File Size and Type Limitations

[JessicaMarine]

What file size limitations (if any) should the Court impose on external filers? 500MB (#1166) Are there any common work-arounds (i.e., upload as multiple docs)? Post-MVP (#1165) Should the court impose file type limitations? Yes, only PDFs (#1164)

Upload Scenarios:

• External initial filings (petition package)
• External subsequent filings
• Court uploads

Issues:

• System dealing with large file sizes, timeout, etc
• Usability of large documents
• Limiting file types
• Workarounds

Solution:

• Pick a max file size and apply across the board
• Limit to PDFs; esp. for PDF. We're only building for PDFs right now.
• Workaround for files that are too big: Users can upload multiple docs.
• Also add helper text to guide users.

Notes/Considerations:

• Don't make so small that frustrating for users
• Any time we ask an external user to upload something, include file size limitations in helptext (make global)
• Don't need to use page count; it doesn't impact file size

Tasks

• [x] Court to test out creating and handling of large docs to see how large of doc they can handle. [Jessica to complete when shutdown has ended]
• [x] Jessica to ask Tony to inquire with AO office
• [x] Stephanie to look into what's been recommended in the past.
• [x] Establish file size limitations

[waldoj]

There needs to be some file size limitation. I'm not sure if it's even possible to have no limit for HTTP, but it's certainly undesirable — a bug could use up many gigabytes of storage before it was noticed. A review of over a million court documents as PDFs yielded none larger than 16 MB, with an average scanned page size at 54k. Note, however, that Alabama Middle District (an apparently random example) limits PDFs to 25M, which may be arbitrary, or perhaps it was based on actual experience.

There is no reason why the limitations can't be as large as is necessary.

Separately, there are page limits that may need to be enforced. This can be done on the client side, using pdf.js. Note, however, that this is a soft limit — we can't force their browser to adhere to this limit, so it remains necessary to have a hard limit coded into the server.

Something that we could reasonably do is scan in some documents at a few different settings and see what we get! Maybe print out a sample court filing and scan it in at a few different settings (black and white, grey scale, and color; TIFF or JPEG; ZIP-compressed and not), and also use a PDF-generation iOS app to do the same. The extremes of sizes that we see may be illustrative.

[JessicaMarine]

Some data from other Federal Courts:

This also provides some helpful information/instructional text: https://www.scd.uscourts.gov/CMECF/Troubleshooting/PageSizeLimits.pdf

• A 10 MB document created through black and white scanning at 300 DPI will be approximately 150-200 pages.
  • A 10 MB document converted to PDF through a word processing program will be approximately 400-500 pages.
  • Based on that logic:

[JessicaMarine]

Recommendation from Court's OIS team:

File size max: 500MB (#1166)

Files larger than 500MB should be broken down into multiple files each with a maximum of 500MB. The system should recognize the files (or volumes) as part of the larger document and make them available for viewing through a single docket entry (not one docket entry for each part). POST-MVP (#1165)

Session Timeouts (due to inactivity): (#837) For external users: 30 minutes. Include visual and audio messages as the timeout approaches requiring action on the user’s part in order to continue without being “timed-out”. For internal users: 1 hour. Include visual and audio messages as the timeout approaches requiring action on the user’s part in order to continue without being “timed-out”. Note: Document upload process should not be considered “activity” as far as the EF CMS system is concerned in relation to timeouts

Virus Scanning (#1164) In the existing CMS eFiled documents go through a virus scanning process in addition to an OCR process. Virus scanning should also be part of the new CMS.