Closed kylekirkby closed 2 years ago
Also... just noticed this block of code in the input lambda:
let mcClient = new AWS.MediaConvert();
if (!AWS.config.mediaconvert) {
try {
const endpoints = await mcClient.describeEndpoints().promise();
AWS.config.mediaconvert = { endpoint: endpoints.Endpoints[0].Url };
// Override so config applies
mcClient = new AWS.MediaConvert();
} catch (e) {
console.log(e);
return;
}
}
π€¦ββοΈ
Hi @kylekirkby, if I'm not wrong you should have inside your SNS message the job detail, no ?
What does your message.detail
object contains ?
const path = require('path')
const { AudioAnalysis } = require('/opt/audio-analysis')
exports.handler = async (event, context) => {
const message = JSON.parse(event.Records[0].Sns.Message)
console.log('Status', message.detail.status)
if (message.detail.status === 'COMPLETE') {
try {
const s3uri =
message.detail.outputGroupDetails[1].outputDetails[0]
.outputFilePaths[0] || undefined
if (!s3uri) throw new Error('s3uri is undefined')
const bucket = path.parse(s3uri).dir.split('/')[2]
const tag = path.parse(s3uri).dir.split('/')[3]
const { transcribe } = new AudioAnalysis({
tag,
mediaFileUri: s3uri,
outputBucket: bucket,
vocabularyFilterName: process.env.VOCABULARY_FILTER_NAME,
outputKey: `${tag}/transcript/`,
mediaFormat: 'mp3',
})
await transcribe.startJob()
} catch (error) {
if (error.name === "ConflictException") {
console.info(error)
return;
}
console.error(error)
}
}
return message
}
Arhhh! Thanks @nathanagez! I've just noticed inputDetails
in the detail section..
detail: {
timestamp: 1633534485958,
accountId: '120662376178',
queue: 'arn:aws:mediaconvert:us-east-1:120662376178:queues/Default',
jobId: 'xxxxxxx,
status: 'INPUT_INFORMATION',
userMetadata: { videoId: '953942b4-7cd8-44a4-bfcb-ed88e7c4d31c' },
inputDetails: [ [Object] ]
}
@nathanagez - on another note, I'm not seeing outputGroupDetails
in my SNS message?
@nathanagez - I'm also transcribing my videos but I'm using the input mp4 video file. Are you using the audio-only version that MediaConvert creates?
@nathanagez - on another note, I'm not seeing
outputGroupDetails
in my SNS message?
I think it depends of your MediaConvert job template, I'm using a custom template where I have output groups π€
@nathanagez - on another note, I'm not seeing
outputGroupDetails
in my SNS message?I think it depends of your MediaConvert job template, I'm using a custom template where I have output groups π€
Hmmm.. strange! I'm also using a custom template (based on default one) with output groups π€. I've added an output group for poster images
SettingsJson:
OutputGroups:
- Name: Apple HLS
Outputs:
- ContainerSettings:
Container: M3U8
M3u8Settings:
AudioFramesPerPes: 4
PcrControl: PCR_EVERY_PES_PACKET
PmtPid: 480
PrivateMetadataPid: 503
ProgramNumber: 1
PatInterval: 0
PmtInterval: 0
Scte35Source: NONE
NielsenId3: NONE
TimedMetadata: NONE
VideoPid: 481
AudioPids:
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
VideoDescription:
ScalingBehavior: DEFAULT
TimecodeInsertion: DISABLED
AntiAlias: ENABLED
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
NumberReferenceFrames: 3
Syntax: DEFAULT
Softness: 0
FramerateDenominator: 1
GopClosedCadence: 1
GopSize: 90
Slices: 1
GopBReference: DISABLED
HrdBufferSize: 4500000
MaxBitrate: 3000000
SlowPal: DISABLED
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
EntropyEncoding: CABAC
FramerateControl: SPECIFIED
RateControlMode: QVBR
QvbrSettings:
QvbrQualityLevel: 9
QvbrQualityLevelFineTune: 0
CodecProfile: MAIN
Telecine: NONE
FramerateNumerator: 30
MinIInterval: 0
AdaptiveQuantization: HIGH
CodecLevel: AUTO
FieldEncoding: PAFF
SceneChangeDetect: ENABLED
QualityTuningLevel: SINGLE_PASS
FramerateConversionAlgorithm: DUPLICATE_DROP
UnregisteredSeiTimecode: DISABLED
GopSizeUnits: FRAMES
ParControl: INITIALIZE_FROM_SOURCE
NumberBFramesBetweenReferenceFrames: 1
RepeatPps: DISABLED
DynamicSubGop: STATIC
AfdSignaling: NONE
DropFrameTimecode: ENABLED
RespondToAfd: NONE
ColorMetadata: INSERT
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
AudioDescriptionBroadcasterMix: NORMAL
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
RawFormat: NONE
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputSettings:
HlsSettings:
AudioGroupId: program_audio
AudioOnlyContainer: AUTOMATIC
IFrameOnlyManifest: EXCLUDE
SegmentModifier: $dt$
NameModifier: _3000
- ContainerSettings:
Container: M3U8
M3u8Settings:
AudioFramesPerPes: 4
PcrControl: PCR_EVERY_PES_PACKET
PmtPid: 480
PrivateMetadataPid: 503
ProgramNumber: 1
PatInterval: 0
PmtInterval: 0
Scte35Source: NONE
NielsenId3: NONE
TimedMetadata: NONE
TimedMetadataPid: 502
VideoPid: 481
AudioPids:
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
VideoDescription:
Width: 960
ScalingBehavior: DEFAULT
Height: 540
TimecodeInsertion: DISABLED
AntiAlias: ENABLED
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
NumberReferenceFrames: 3
Syntax: DEFAULT
Softness: 0
FramerateDenominator: 1
GopClosedCadence: 1
GopSize: 90
Slices: 1
GopBReference: DISABLED
HrdBufferSize: 2250000
MaxBitrate: 1500000
SlowPal: DISABLED
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
EntropyEncoding: CABAC
FramerateControl: SPECIFIED
RateControlMode: QVBR
QvbrSettings:
QvbrQualityLevel: 8
QvbrQualityLevelFineTune: 0
CodecProfile: MAIN
Telecine: NONE
FramerateNumerator: 30
MinIInterval: 0
AdaptiveQuantization: HIGH
CodecLevel: AUTO
FieldEncoding: PAFF
SceneChangeDetect: ENABLED
QualityTuningLevel: SINGLE_PASS
FramerateConversionAlgorithm: DUPLICATE_DROP
UnregisteredSeiTimecode: DISABLED
GopSizeUnits: FRAMES
ParControl: INITIALIZE_FROM_SOURCE
NumberBFramesBetweenReferenceFrames: 1
RepeatPps: DISABLED
DynamicSubGop: STATIC
AfdSignaling: NONE
DropFrameTimecode: ENABLED
RespondToAfd: NONE
ColorMetadata: INSERT
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
AudioDescriptionBroadcasterMix: NORMAL
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
RawFormat: NONE
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputSettings:
HlsSettings:
AudioGroupId: program_audio
AudioOnlyContainer: AUTOMATIC
IFrameOnlyManifest: EXCLUDE
SegmentModifier: $dt$
NameModifier: _1500
- ContainerSettings:
Container: M3U8
M3u8Settings:
AudioFramesPerPes: 4
PcrControl: PCR_EVERY_PES_PACKET
PmtPid: 480
PrivateMetadataPid: 503
ProgramNumber: 1
PatInterval: 0
PmtInterval: 0
Scte35Source: NONE
NielsenId3: NONE
TimedMetadata: NONE
TimedMetadataPid: 502
VideoPid: 481
AudioPids:
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
VideoDescription:
Width: 768
ScalingBehavior: DEFAULT
Height: 432
TimecodeInsertion: DISABLED
AntiAlias: ENABLED
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
NumberReferenceFrames: 3
Syntax: DEFAULT
Softness: 0
FramerateDenominator: 1
GopClosedCadence: 1
GopSize: 90
Slices: 1
GopBReference: DISABLED
HrdBufferSize: 1250000
MaxBitrate: 750000
SlowPal: DISABLED
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
EntropyEncoding: CABAC
FramerateControl: SPECIFIED
RateControlMode: QVBR
QvbrSettings:
QvbrQualityLevel: 7
QvbrQualityLevelFineTune: 0
CodecProfile: MAIN
Telecine: NONE
FramerateNumerator: 30
MinIInterval: 0
AdaptiveQuantization: HIGH
CodecLevel: AUTO
FieldEncoding: PAFF
SceneChangeDetect: ENABLED
QualityTuningLevel: SINGLE_PASS
FramerateConversionAlgorithm: DUPLICATE_DROP
UnregisteredSeiTimecode: DISABLED
GopSizeUnits: FRAMES
ParControl: INITIALIZE_FROM_SOURCE
NumberBFramesBetweenReferenceFrames: 2
RepeatPps: DISABLED
DynamicSubGop: STATIC
AfdSignaling: NONE
DropFrameTimecode: ENABLED
RespondToAfd: NONE
ColorMetadata: INSERT
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
AudioDescriptionBroadcasterMix: NORMAL
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
RawFormat: NONE
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputSettings:
HlsSettings:
AudioGroupId: program_audio
AudioOnlyContainer: AUTOMATIC
IFrameOnlyManifest: EXCLUDE
SegmentModifier: $dt$
NameModifier: _750
- ContainerSettings:
Container: M3U8
M3u8Settings:
AudioFramesPerPes: 4
PcrControl: PCR_EVERY_PES_PACKET
PmtPid: 480
PrivateMetadataPid: 503
ProgramNumber: 1
PatInterval: 0
PmtInterval: 0
Scte35Source: NONE
NielsenId3: NONE
TimedMetadata: NONE
VideoPid: 481
AudioPids:
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
VideoDescription:
Width: 640
ScalingBehavior: DEFAULT
Height: 360
TimecodeInsertion: DISABLED
AntiAlias: ENABLED
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
NumberReferenceFrames: 3
Syntax: DEFAULT
Softness: 0
FramerateDenominator: 1
GopClosedCadence: 1
GopSize: 90
Slices: 1
GopBReference: DISABLED
HrdBufferSize: 450000
MaxBitrate: 325000
SlowPal: DISABLED
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
EntropyEncoding: CABAC
FramerateControl: SPECIFIED
RateControlMode: QVBR
QvbrSettings:
QvbrQualityLevel: 6
QvbrQualityLevelFineTune: 0
CodecProfile: MAIN
Telecine: NONE
FramerateNumerator: 30
MinIInterval: 0
AdaptiveQuantization: HIGH
CodecLevel: AUTO
FieldEncoding: PAFF
SceneChangeDetect: ENABLED
QualityTuningLevel: SINGLE_PASS
FramerateConversionAlgorithm: DUPLICATE_DROP
UnregisteredSeiTimecode: DISABLED
GopSizeUnits: FRAMES
ParControl: INITIALIZE_FROM_SOURCE
NumberBFramesBetweenReferenceFrames: 2
RepeatPps: DISABLED
DynamicSubGop: STATIC
AfdSignaling: NONE
DropFrameTimecode: ENABLED
RespondToAfd: NONE
ColorMetadata: INSERT
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
AudioDescriptionBroadcasterMix: NORMAL
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
RawFormat: NONE
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputSettings:
HlsSettings:
AudioGroupId: program_audio
AudioOnlyContainer: AUTOMATIC
IFrameOnlyManifest: EXCLUDE
SegmentModifier: $dt$
NameModifier: _325
- ContainerSettings:
Container: M3U8
M3u8Settings:
AudioFramesPerPes: 4
PcrControl: PCR_EVERY_PES_PACKET
PmtPid: 480
PrivateMetadataPid: 503
ProgramNumber: 1
PatInterval: 0
PmtInterval: 0
Scte35Source: NONE
NielsenId3: NONE
TimedMetadata: NONE
TimedMetadataPid: 502
VideoPid: 481
AudioPids:
- 482
- 483
- 484
- 485
- 486
- 487
- 488
- 489
- 490
- 491
- 492
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
AudioDescriptionBroadcasterMix: NORMAL
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
RawFormat: NONE
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputSettings:
HlsSettings:
AudioGroupId: program_audio
AudioOnlyContainer: AUTOMATIC
IFrameOnlyManifest: EXCLUDE
SegmentModifier: $dt$
NameModifier: _audio
OutputGroupSettings:
Type: HLS_GROUP_SETTINGS
HlsGroupSettings:
ManifestDurationFormat: INTEGER
SegmentLength: 10
TimedMetadataId3Period: 10
CaptionLanguageSetting: OMIT
TimedMetadataId3Frame: PRIV
CodecSpecification: RFC_4281
OutputSelection: MANIFESTS_AND_SEGMENTS
ProgramDateTimePeriod: 600
MinSegmentLength: 0
MinFinalSegmentLength: 0
DirectoryStructure: SINGLE_DIRECTORY
ProgramDateTime: EXCLUDE
SegmentControl: SEGMENTED_FILES
ManifestCompression: NONE
ClientCache: ENABLED
StreamInfResolution: INCLUDE
- Name: File Group
CustomName: Poster Images
Outputs:
- ContainerSettings:
Container: RAW
VideoDescription:
CodecSettings:
Codec: FRAME_CAPTURE
FrameCaptureSettings:
FramerateNumerator: 30
FramerateDenominator: 150
MaxCaptures: 3
Quality: 80
Extension: .jpg
NameModifier: poster-image
OutputGroupSettings:
Type: FILE_GROUP_SETTINGS
AdAvailOffset: 0
@nathanagez - outputGroupDetails
only exists when status is set to "COMPLETE" ππΌ
Is your feature request related to a problem? Please describe. The Media Convert status lambda currently returns a MediaConvert job ID. This on its own is not very useful. Even when submitting the job to MediaConvert you don't get returned a job ID to match to the status events. I was hoping to call mediaConvert.getJob with the job ID to get the name of the file that the job relates to (with this I could then update the DynamoDB entry for the video with the conversion progress).
I'm currently getting this error message when calling
getJob()
.How can I get the "custom specific" endpoint? I'd put this in by hand but I'd rather it be dynamic if possible due to IaaC principles etc.
Describe the solution you'd like Some guidance on how to access this endpoint within the MediaConvert status lambda. It would also be good to add this by default as, AFAIK, the MediaConvert response is useless without some way of correlating the status to a specific video.
Describe alternatives you've considered I've added the region environment variable in hope that this would resolve the issue to no avail.
As I'm writing this issue I've noticed a
userMetadata: {},
attribute is returned in the SNS event message. Maybe when creating the MC job I could tag the job with the video ID. I'll update this ticket if I can get that working.Cheers!