Enabler: Store generated page in an S3 bucket

[blairlearn]

Refactor to replace MS SQL Server with an S3 bucket

• Continue to generate via an IIS site
• Instead of creating a job to clean up old documents, set an 'Expires' header with a suitable offset and let AWS do the work.
• Configuration items:
  • RetentionPeriod - (integer) The number of days from generation before the document is deleted.
  • BucketName - The name of the bucket where generated pages are stored.
  • Region??? - Not clear if this is needed separately, or just part of the bucket name. (I think the latter)
  • Presumably some sort of credentials, presumably an access key and a secret key which will be stored in the web.config.
• Update Launch URL

Clean up expired documents

Instead of creating a job to clean up old documents, set an 'Expires' header with a suitable offset and let AWS do the work.

• https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/MS3PutObjectPutObjectRequest.html
• https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/THeadersCollection.html
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-expire-general-considerations.html

Code Changes

• Configuration items
  • RetentionPeriod - (integer) The number of days from generation before the document is deleted.
  • BucketName - The name of the bucket where generated pages are stored.
  • Some sort of credentials ???? - Access Key and Secret key??? How to connect??
  • Dev environment
    • localstack requires Docker (not available on VM)
    • CTS print svc is .Net 4.8
    • Castle.NVelocity nuget is full framework
    • NVelocity (No namespace) is also Castle, but an unknown
• CTSPrintRequestHandler
  • GenerateCachedPage()
    • Move link embedding logic from GetCachedContent()
      • Generate cacheID instead of retrieving from the data store.
      • Replace ${generatePrintURL} in the document body with the reference URL.
    • Make reference URL a configurable format string.
    • Pass cacheID to PrintCacheManager::Save()
  • GetCachedContent()
    • Move logic to dynamically insert a reference URL to GenerateCachedPage().
    • Remove legacy logic to dynamically insert JavaScript for mailto: link. (This is now part of the page template.)

• PrintCacheManager

  • Remove database connection string.

  • Save()

    • Receieve cacheID and save as object Key parameter.
    • Convert trialIDs list and searchCriteria object to strings.
    • Save string version of trialIDs list and searchCriteria object as metadata via PutObjectRequest() call

    • Replace the body of the method with code along the lines of

      // Create a client
      AmazonS3Client client = new AmazonS3Client();
      
      // Preserve metadata
      MetadataCollection metadata = new MetadataCollection();
      Add("trial-id-list", trialIDString);
      Add("search-criteria", searchCriteriaString)
      
      // Create a PutObject request
      PutObjectRequest request = new PutObjectRequest
      {
          BucketName = "bucket-name",
          Key = "public-guid-like-what-we-have-now",
          ContentBody = "Big blob of HTML",
          ContentType = "text/html",
          Metadata = metadata
      };
      
      // Put object
      PutObjectResponse response = client.PutObject(request);

      • The bucket name, and likely some parameters to the AmazonS3Client constructor come from a configuration file.

  • GetPage()

    • Use GetObject()/GetObjectRequest to retrieve the document.

    • Convert response to string. Something like

      // Create a client
      AmazonS3Client client = new AmazonS3Client();
      
      // Create a GetObject request
      GetObjectRequest request = new GetObjectRequest
      {
          BucketName = "bucket-name",
          Key = "public-guid-like-what-we-have-now"
      };
      
      // Issue request and remember to dispose of the response
      using (GetObjectResponse response = client.GetObject(request))
      {
          using (StreamReader reader = new StreamReader(response.ResponseStream))
          {
              string contents = reader.ReadToEnd();
          }
      }

      • The bucket name, and likely some parameters to the AmazonS3Client constructor come from a configuration file.
  • CreatePrintIdDataTable()
    • Remove this method.

ESTIMATE TBD

Resources:

Prerequisites

Sub-Tasks

Notes