Discovery: Script to move unattached PDFs from Drupal into S3 Bucket

[gracekretschmer-metrostar]

User Story or Problem Statement

The CMS team needs to understand what is feasible and the process to remove inactive PDFs from Drupal and into a S3 bucket and what options do we have to limit the types of PDFs that get removed.

Description or Additional Context

Right now, there are unattached PDFs within Drupal CMS that need to be moved out of Drupal and into a S3 bucket. An unattached PDF is a PDF that is not connect to any content within Drupal CMS. The CMS team has been working with VHA Digital Media and have agreed to the idea of creating a script to clear out any unused PDFs after a specific # of days. We also need the PDFs that are moved into the S3 bucket to be easily searchable, in case a editor needs the PDF or for a FOIA request.

Before we move forward with implementing the script, we want to create a plan of action.

Relevant Links

• https://app.mural.co/t/departmentofveteransaffairs9999/m/departmentofveteransaffairs9999/1717188891528/e97a85414623fb12c9e927af476f8cda158cdd2b?sender=u4c8b6076357da42993941037

Steps for Implementation

• Discovery to understand how feasible it to limit the script to unattached PDFs for only the VHA Digital Media sections.
• Create a script to identify the unattached PDFs
• Create a script to remove unattached PDFs from Drupal and into a S3 bucket
• Generate a log file of the unattached PDFs that were removed from Drupal and include metadata about the PDFs (e.g. File Name, Upload Date, Uploaded By).
• Limit the access to the S3 bucket to only CMS engineers
• Discovery to determine how to run the script on a regular cadence (which is to be determined)

Anti-Goals

• Do not run script until we've received a sign off from VHA Digital Media
• A user cannot search for specific PDFs within the S3 bucket.

Stretch Goal

• Run a script on a test PDF in a test environment to confirm that the PDF

Acceptance Criteria

• [ ] An understanding of if/how we can limit the types of PDFs that can get removed from Drupal.
• [ ] A plan of action to remove PDFs from Drupal and into a S3 bucket.
• [ ] The PDFs within the S3 bucket should include enough metadata to be searchable by a user

[anantais]

Current progress:

Installed this module that Edmund had recommended using: https://www.drupal.org/project/s3fs Looking at how to configure this. I will likely be setting up a meeting with Tyler in the next day or so to discuss it.

[anantais]

What is the "VHA Digital Media sections" referring to? Is that any pdf that falls under the VHA section? Is there a specific tag or something else being used to identify these pdfs?

^ This question was answered in Slack

[michelle-dooley]

Drupal devs cannot get access AWS. We need to determine next best path to move forward with this work.

Note: this issue has been resolved we should be able to continue as planned

[7hunderbird]

@michelle-dooley I appologize if I hadn't communicated this directly before, but we're working on getting Amanda and Jake a "developer" access to AWS.