Save edits throws an error in Thumbnails operator in v2.0.0

[aburkleaux-amazon]

Describe the bug

When I click on the Save edits button after editing subtitles, the workflow starts to rerun but ends with an Error status.

To Reproduce

1. Upload a video and run a workflow. The workflow I ran had the checkboxes for all operators enabled.
2. Edit the subtitles for the workflow
3. Click Save edits
4. The workflow ends in an Error state. The Step Functions console shows the error is in the Thumbnails operator

Expected behavior

The workflow reruns starting from the Translate operator. I do not expect Thumbnails or any other operator up to Transcribe to be rerun.

Please complete the following information about the solution:

• [ ] Version: 2.0.0

• [ ] Region: us-east-1

• [ ] Was the solution modified from the version published on this repository?

No

• [ ] If the answer to the previous question was yes, are the changes available on GitHub?

• [ ] Have you checked your service quotas for the sevices this solution uses?

Yes

• [ ] Were there any errors in the CloudWatch Logs?

This is the error from the Step Output

{
  "Name": "Thumbnail",
  "AssetId": "1ab4bf93-0479-496f-954a-e29c2d96eb61",
  "WorkflowExecutionId": "f15cc3bf-bc93-43c5-a437-3cfc62c7bf4b",
  "Input": {
    "Media": {
      "Video": {
        "S3Bucket": "clo2-miestack-XXXXXXX-dataplane-XXXXXXXX",
        "S3Key": "public/upload/TWAN-Spanish-Sept18-2020~medium.mp4"
      }
    },
    "MetaData": {}
  },
  "Configuration": {
    "MediaType": "Video",
    "Enabled": true,
    "ThumbnailPosition": "10"
  },
  "Status": "Executing",
  "MetaData": {
    "MediaconvertJobId": "XXXXXXXXXXX",
    "MediaconvertInputFile": "s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXX/public/upload/TWAN-Spanish-Sept18-2020~medium.mp4",
    "AssetId": "1ab4bf93-0479-496f-954a-e29c2d96eb61",
    "WorkflowExecutionId": "f15cc3bf-bc93-43c5-a437-3cfc62c7bf4b"
  },
  "Media": {},
  "Outputs": {
    "Error": "MasExecutionError",
    "Cause": "{\"errorMessage\": \"{'Name': 'Thumbnail', 'AssetId': '1ab4bf93-0479-496f-954a-e29c2d96eb61', 'WorkflowExecutionId': 'f15cc3bf-bc93-43c5-a437-3cfc62c7bf4b', 'Input': {'Media': {'Video': {'S3Bucket': 'clo2-miestack-XXXXXXX-dataplane-XXXXXXXX', 'S3Key': 'public/upload/TWAN-Spanish-Sept18-2020~medium.mp4'}}, 'MetaData': {}}, 'Configuration': {'MediaType': 'Video', 'Enabled': True, 'ThumbnailPosition': '10'}, 'Status': 'Error', 'MetaData': {'MediaconvertJobId': 'XXXXXXXX', 'MediaconvertInputFile': 's3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXXpublic/upload/TWAN-Spanish-Sept18-2020~medium.mp4', 'AssetId': '1ab4bf93-0479-496f-954a-e29c2d96eb61', 'WorkflowExecutionId': 'f15cc3bf-bc93-43c5-a437-3cfc62c7bf4b', 'MediaconvertError': 'Unhandled exception, unable to get status from mediaconvert: {\\\\'ResponseMetadata\\\\': {\\\\'RequestId\\\\': \\\\'432f18e4-67fa-41cc-a0b2-9fdaabee8ee5\\\\', \\\\'HTTPStatusCode\\\\': 200, \\\\'HTTPHeaders\\\\': {\\\\'content-type\\\\': \\\\'application/json\\\\', \\\\'content-length\\\\': \\\\'4737\\\\', \\\\'connection\\\\': \\\\'keep-alive\\\\', \\\\'date\\\\': \\\\'Wed, 30 Mar 2022 23:52:39 GMT\\\\', \\\\'x-amzn-requestid\\\\': \\\\'432f18e4-67fa-41cc-a0b2-9fdaabee8ee5\\\\', \\\\'x-amz-apigw-id\\\\': \\\\'P0oDLHmWIAMFouw=\\\\', \\\\'x-amzn-trace-id\\\\': \\\\'Root=1-6244ed2f-2cab5fc0598d7c057cbfa48c\\\\', \\\\'x-cache\\\\': \\\\'Miss from cloudfront\\\\', \\\\'via\\\\': \\\\'1.1 a0042b5b0abdf83ac753e9d8be59e4e4.cloudfront.net (CloudFront)\\\\', \\\\'x-amz-cf-pop\\\\': \\\\'IAD79-C3\\\\', \\\\'x-amz-cf-id\\\\': \\\\'JWcHDEedHWwqdB0IMZIVPx1shX47K0dyHlHGtVtKLT0pU4FKl8F0jA==\\\\'}, \\\\'RetryAttempts\\\\': 0}, \\\\'Job\\\\': {\\\\'AccelerationSettings\\\\': {\\\\'Mode\\\\': \\\\'DISABLED\\\\'}, \\\\'AccelerationStatus\\\\': \\\\'NOT_APPLICABLE\\\\', \\\\'Arn\\\\': \\\\'arn:aws:mediaconvert:us-east-1:526662735483:jobs/XXXXXXX\\\\', \\\\'CreatedAt\\\\': datetime.datetime(2022, 3, 30, 23, 52, 19, tzinfo=tzlocal()), \\\\'ErrorCode\\\\': 1401, \\\\'ErrorMessage\\\\': \\\"Unable to open input file [s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXXpublic/upload/TWAN-Spanish-Sept18-2020~medium.mp4]: [Failed probe/open: [Can\\\\'t read input stream: [Failed to read data: Access denied to s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXX/public/upload/TWAN-Spanish-Sept18-2020~medium.mp4]]]\\\", \\\\'Id\\\\': \\\\'XXXXXXXXXXX\\\\', \\\\'Messages\\\\': {\\\\'Info\\\\': [], \\\\'Warning\\\\': []}, \\\\'Priority\\\\': 0, \\\\'Queue\\\\': \\\\'arn:aws:mediaconvert:us-east-1:526662735483:queues/Default\\\\', \\\\'Role\\\\': \\\\'arn:aws:iam::526662735483:role/clo2-MieStack-XXXXXXX-Ope-mediaConvertS3Role-1XSBDW6C9UV7Y\\\\', \\\\'Settings\\\\': {\\\\'Inputs\\\\': [{\\\\'AudioSelectors\\\\': {\\\\'Audio Selector 1\\\\': {\\\\'DefaultSelection\\\\': \\\\'DEFAULT\\\\', \\\\'Offset\\\\': 0, \\\\'ProgramSelection\\\\': 1}}, \\\\'FileInput\\\\': \\\\'s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXX/public/upload/TWAN-Spanish-Sept18-2020~medium.mp4\\\\', \\\\'VideoSelector\\\\': {\\\\'ColorSpace\\\\': \\\\'FOLLOW\\\\'}}], \\\\'OutputGroups\\\\': [{\\\\'CustomName\\\\': \\\\'thumbnail\\\\', \\\\'Name\\\\': \\\\'File Group\\\\', \\\\'OutputGroupSettings\\\\': {\\\\'FileGroupSettings\\\\': {\\\\'Destination\\\\': \\\\'s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXX/private/assets/1ab4bf93-0479-496f-954a-e29c2d96eb61/\\\\'}, \\\\'Type\\\\': \\\\'FILE_GROUP_SETTINGS\\\\'}, \\\\'Outputs\\\\': [{\\\\'ContainerSettings\\\\': {\\\\'Container\\\\': \\\\'RAW\\\\'}, \\\\'Extension\\\\': \\\\'jpg\\\\', \\\\'NameModifier\\\\': \\\\'_thumbnail\\\\', \\\\'VideoDescription\\\\': {\\\\'AntiAlias\\\\': \\\\'ENABLED\\\\', \\\\'CodecSettings\\\\': {\\\\'Codec\\\\': \\\\'FRAME_CAPTURE\\\\', \\\\'FrameCaptureSettings\\\\': {\\\\'FramerateDenominator\\\\': 10, \\\\'FramerateNumerator\\\\': 1, \\\\'MaxCaptures\\\\': 2, \\\\'Quality\\\\': 80}}, \\\\'ColorMetadata\\\\': \\\\'INSERT\\\\', \\\\'DropFrameTimecode\\\\': \\\\'ENABLED\\\\', \\\\'ScalingBehavior\\\\': \\\\'DEFAULT\\\\', \\\\'Sharpness\\\\': 50, \\\\'TimecodeInsertion\\\\': \\\\'DISABLED\\\\'}}]}, {\\\\'Name\\\\': \\\\'File Group\\\\', \\\\'OutputGroupSettings\\\\': {\\\\'FileGroupSettings\\\\': {\\\\'Destination\\\\': \\\\'s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXXprivate/assets/1ab4bf93-0479-496f-954a-e29c2d96eb61/workflows/f15cc3bf-bc93-43c5-a437-3cfc62c7bf4b/\\\\'}, \\\\'Type\\\\': \\\\'FILE_GROUP_SETTINGS\\\\'}, \\\\'Outputs\\\\': [{\\\\'AudioDescriptions\\\\': [{\\\\'AudioSourceName\\\\': \\\\'Audio Selector 1\\\\', \\\\'AudioTypeControl\\\\': \\\\'FOLLOW_INPUT\\\\', \\\\'CodecSettings\\\\': {\\\\'AacSettings\\\\': {\\\\'AudioDescriptionBroadcasterMix\\\\': \\\\'NORMAL\\\\', \\\\'Bitrate\\\\': 96000, \\\\'CodecProfile\\\\': \\\\'LC\\\\', \\\\'CodingMode\\\\': \\\\'CODING_MODE_2_0\\\\', \\\\'RateControlMode\\\\': \\\\'CBR\\\\', \\\\'RawFormat\\\\': \\\\'NONE\\\\', \\\\'SampleRate\\\\': 48000, \\\\'Specification\\\\': \\\\'MPEG4\\\\'}, \\\\'Codec\\\\': \\\\'AAC\\\\'}, \\\\'LanguageCodeControl\\\\': \\\\'FOLLOW_INPUT\\\\'}], \\\\'ContainerSettings\\\\': {\\\\'Container\\\\': \\\\'MP4\\\\', \\\\'Mp4Settings\\\\': {\\\\'CslgAtom\\\\': \\\\'INCLUDE\\\\', \\\\'FreeSpaceBox\\\\': \\\\'EXCLUDE\\\\', \\\\'MoovPlacement\\\\': \\\\'PROGRESSIVE_DOWNLOAD\\\\'}}, \\\\'Extension\\\\': \\\\'mp4\\\\', \\\\'NameModifier\\\\': \\\\'_audio\\\\'}]}, {\\\\'CustomName\\\\': \\\\'proxy\\\\', \\\\'Name\\\\': \\\\'File Group\\\\', \\\\'OutputGroupSettings\\\\': {\\\\'FileGroupSettings\\\\': {\\\\'Destination\\\\': \\\\'s3://clo2-miestack-XXXXXXX-dataplane-XXXXXXXX/private/assets/1ab4bf93-0479-496f-954a-e29c2d96eb61/\\\\'}, \\\\'Type\\\\': \\\\'FILE_GROUP_SETTINGS\\\\'}, \\\\'Outputs\\\\': [{\\\\'AudioDescriptions\\\\': [{\\\\'AudioSourceName\\\\': \\\\'Audio Selector 1\\\\', \\\\'AudioTypeControl\\\\': \\\\'FOLLOW_INPUT\\\\', \\\\'CodecSettings\\\\': {\\\\'AacSettings\\\\': {\\\\'AudioDescriptionBroadcasterMix\\\\': \\\\'NORMAL\\\\', \\\\'Bitrate\\\\': 64000, \\\\'CodecProfile\\\\': \\\\'LC\\\\', \\\\'CodingMode\\\\': \\\\'CODING_MODE_2_0\\\\', \\\\'RateControlMode\\\\': \\\\'CBR\\\\', \\\\'RawFormat\\\\': \\\\'NONE\\\\', \\\\'SampleRate\\\\': 48000, \\\\'Specification\\\\': \\\\'MPEG4\\\\'}, \\\\'Codec\\\\': \\\\'AAC\\\\'}, \\\\'LanguageCodeControl\\\\': \\\\'FOLLOW_INPUT\\\\'}], \\\\'ContainerSettings\\\\': {\\\\'Container\\\\': \\\\'MP4\\\\', \\\\'Mp4Settings\\\\': {\\\\'CslgAtom\\\\': \\\\'INCLUDE\\\\', \\\\'FreeSpaceBox\\\\': \\\\'EXCLUDE\\\\', \\\\'MoovPlacement\\\\': \\\\'PROGRESSIVE_DOWNLOAD\\\\'}}, \\\\'Extension\\\\': \\\\'mp4\\\\', \\\\'NameModifier\\\\': \\\\'_proxy\\\\', \\\\'VideoDescription\\\\': {\\\\'AfdSignaling\\\\': \\\\'NONE\\\\', \\\\'AntiAlias\\\\': \\\\'ENABLED\\\\', \\\\'CodecSettings\\\\': {\\\\'Codec\\\\': \\\\'H_264\\\\', \\\\'H264Settings\\\\': {\\\\'AdaptiveQuantization\\\\': \\\\'HIGH\\\\', \\\\'Bitrate\\\\': 1600000, \\\\'CodecLevel\\\\': \\\\'AUTO\\\\', \\\\'CodecProfile\\\\': \\\\'MAIN\\\\', \\\\'EntropyEncoding\\\\': \\\\'CABAC\\\\', \\\\'FieldEncoding\\\\': \\\\'PAFF\\\\', \\\\'FlickerAdaptiveQuantization\\\\': \\\\'DISABLED\\\\', \\\\'FramerateControl\\\\': \\\\'SPECIFIED\\\\', \\\\'FramerateConversionAlgorithm\\\\': \\\\'DUPLICATE_DROP\\\\', \\\\'FramerateDenominator\\\\': 1, \\\\'FramerateNumerator\\\\': 30, \\\\'GopBReference\\\\': \\\\'DISABLED\\\\', \\\\'GopClosedCadence\\\\': 1, \\\\'GopSize\\\\': 90.0, \\\\'GopSizeUnits\\\\': \\\\'FRAMES\\\\', \\\\'InterlaceMode\\\\': \\\\'PROGRESSIVE\\\\', \\\\'MinIInterval\\\\': 0, \\\\'NumberBFramesBetweenReferenceFrames\\\\': 2, \\\\'NumberReferenceFrames\\\\': 3, \\\\'ParControl\\\\': \\\\'SPECIFIED\\\\', \\\\'ParDenominator\\\\': 1, \\\\'ParNumerator\\\\': 1, \\\\'QualityTuningLevel\\\\': \\\\'SINGLE_PASS\\\\', \\\\'RateControlMode\\\\': \\\\'CBR\\\\', \\\\'RepeatPps\\\\': \\\\'DISABLED\\\\', \\\\'SceneChangeDetect\\\\': \\\\'ENABLED\\\\', \\\\'Slices\\\\': 1, \\\\'SlowPal\\\\': \\\\'DISABLED\\\\', \\\\'Softness\\\\': 0, \\\\'SpatialAdaptiveQuantization\\\\': \\\\'ENABLED\\\\', \\\\'Syntax\\\\': \\\\'DEFAULT\\\\', \\\\'Telecine\\\\': \\\\'NONE\\\\', \\\\'TemporalAdaptiveQuantization\\\\': \\\\'ENABLED\\\\', \\\\'UnregisteredSeiTimecode\\\\': \\\\'DISABLED\\\\'}}, \\\\'ColorMetadata\\\\': \\\\'INSERT\\\\', \\\\'DropFrameTimecode\\\\': \\\\'ENABLED\\\\', \\\\'RespondToAfd\\\\': \\\\'NONE\\\\', \\\\'ScalingBehavior\\\\': \\\\'DEFAULT\\\\', \\\\'Sharpness\\\\': 50, \\\\'TimecodeInsertion\\\\': \\\\'DISABLED\\\\'}}]}]}, \\\\'Status\\\\': \\\\'ERROR\\\\', \\\\'StatusUpdateInterval\\\\': \\\\'SECONDS_60\\\\', \\\\'Timing\\\\': {\\\\'FinishTime\\\\': datetime.datetime(2022, 3, 30, 23, 52, 24, tzinfo=tzlocal()), \\\\'StartTime\\\\': datetime.datetime(2022, 3, 30, 23, 52, 23, tzinfo=tzlocal()), \\\\'SubmitTime\\\\': datetime.datetime(2022, 3, 30, 23, 52, 19, tzinfo=tzlocal())}, \\\\'UserMetadata\\\\': {}}}'}, 'Media': {}}\", \"errorType\": \"MasExecutionError\", \"requestId\": \"eb92124c-bb3f-4228-a0b8-904a22843b0b\", \"stackTrace\": [\"  File \\\"/var/task/check_thumbnail.py\\\", line 98, in lambda_handler\\n    raise MasExecutionError(operator_object.return_output_object())\\n\"]}"
  }
}

Screenshots

Additional context

Add any other context about the problem here.

[aburkleaux-amazon]

I updated the priority to "highest" as this is likely to be hit by anyone who tries to edit subtitles and save them. i.e. the happy path is broken.

[aburkleaux-amazon]

I ran a second workflow that only had the default operators enabled (Transcribe, Subtitles, Translate) and it seems to be working.

[aburkleaux-amazon]

I just ran into this error again using the default workflow. The workflow I reprocessed is using a Custom Vocabulary, so maybe that is the trigger.

[aburkleaux-amazon]

And, I just got the same error on a new workflow (started from the Upload page). This workflow also had a Custom Vocabulary. It's possible this problem is intermittent, since I have many successful workflows from Upload.

[ianwow]

So far I've been unable to reproduce this issue. I've validated "Save Edits" when all operators are enabled, with and without custom vocabularies.