AWS MediaLive Cloudformation Fn::GetAtt get array value

[samueleastdev]

I am building a Cloudformation template for Media Live I have everything working with the AWS PHP SDK. I create a MediaPackage channel and return the data to be used in the next call.

Return example.

(
    [Arn] => arn:aws:mediapackage:us-east-1:000000000000:channels/000000000000
    [Description] => Tests
    [HlsIngest] => Array
        (
            [IngestEndpoints] => Array
                (
                    [0] => Array
                        (
                            [Id] => 000000000000
                            [Password] => 000000000000
                            [Url] => https://000000000000.mediapackage.us-east-1.amazonaws.com/in/v2/1e803c424d2947f58c07d9a6a5ff3d31/000000000000/channel
                            [Username] => 000000000000
                        )

                    [1] => Array
                        (
                            [Id] => 000000000000
                            [Password] => 000000000000
                            [Url] => https://2bcff136c2fbf1e5.mediapackage.us-east-1.amazonaws.com/in/v2/000000000000/10392b8387fd442eae56f29ac2656837/channel
                            [Username] => 000000000000
                        )

                )

        )

    [Id] => sdv
    [Tags] => Array
        (
        )
)

I can then use these values in PHP like this.

$destinationOneUrl      = $createChannel['HlsIngest']['IngestEndpoints'][0]['Url'];
$destinationOnePassword = $createChannel['HlsIngest']['IngestEndpoints'][0]['Password'];
$destinationOneUsername = $createChannel['HlsIngest']['IngestEndpoints'][0]['Username'];

And use these when creating the media live channel.

"Destinations": [
            {
              "Id": "destination1",
              "Settings": [
                {
                    "Url": "' . $destinationOneUrl . '",
                    "Username": "' . $destinationOneUsername . '",
                    "PasswordParam": "' . $streamName . '"
                },
              ]
            }
          ],

Which works but with Cloudformation you have to use the Fn::GetAtt like this.

"Destinations": [{
                    "Id": "destination1",
                    "Settings": [{
                            "Url": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                            "Username": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                            "PasswordParam": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                        },
                        {
                            "Url": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                            "Username": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                            "PasswordParam": {
                                "Fn::GetAtt": ["MediaPackageChannel", "HlsIngest"]
                            },
                        }
                    ]
                }],

You cannot go into the array with Fn::GetAtt is there another function that can be used in combination with the Fn::GetAtt to get the inner values.

Something along these lines

"Fn::GetAtt": ["MediaPackageChannel", "HlsIngest['IngestEndpoints'][0]['Url']"]

Here is my full template its hard to explain but basically I need to attach my medialive channel to the mediapackage channel as I mention I have done with the php code above Cloudformation doesn't make this easy.

You will see the Destinations section line 39 I need to populate the Url, Username, PasswordParam with the returned values from the mediapackage channel creation.

https://gist.github.com/samueleastdev/03b050f937855a81f36bb1e3260aa5d3

I started with this template https://docs.aws.amazon.com/solutions/latest/live-streaming/template.html and I am trying to do it myself they use custom lambda functions within the template so maybe it is not possible without lambda.

Any suggestions or help.

[morjoan]

Hi, I'm not sure there is a way to do what you are doing without the use of a Lambda function. However, one thing to consider might be to use the MediaPackageSettings (instead of Settings) for your MediaLive channel's destination (see: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-medialive-channel-outputdestination.html). By doing so, you'll only need to provide the MediaPackage's channel ID and not have to specify the Url, Username, etc. Hope that helps.

[samueleastdev]

Thanks for the response @morjoan I have tried using the MediaPackageSettings and REF the mediachannel

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-medialive-channel-outputdestination.html

"Destinations": [{
                    "Id": "destination1",
                    "MediaPackageSettings": [{
                        "ChannelId" : "MediaPackageChannel"
                    }]
                }]

But this then give you errors about output groups.

output_groups[0].output_group_settings.destination.channel_id instance.destination additionalProperty "channel_id" exists in instance when not allowed; output_groups[0].output_group_settings.destination.uri instance.destination requires property "uri"; output_groups[0].output_group_settings Object does not match discriminator "hls_group"; output_groups[0].output_group_settings.destination.channel_id instance.destination additionalProperty "channel_id" exists in instance when not allowed; output_groups[0].output_group_settings.destination.uri instance.destination requires property "uri"; output_groups[0].output_group_settings Object does not match discriminator "hls_group" (Service: AWSMediaLive; Status Code: 422; Error Code: UnprocessableEntityException; Request ID: 74a4e4bc-3ffd-4ae2-83bc-62b9b342e261; Proxy: null)

I can get the template to run by inserting dummy settings like this.

"Destinations": [{
                    "Id": "destination1",
                    "Settings": [
                        {
                            "Url": "https://my_destination1/testing",
                            "Username": "user",
                            "PasswordParam": "my_destination1"
                        }
                    ]
                }],

Here is the full template that will complete.

https://gist.github.com/samueleastdev/926a0bfbe7b8af193643ddda2ac9cd73

But obviously it is not connected to the mediapackage channel.

Do you know how to get the full code for this template? https://docs.aws.amazon.com/solutions/latest/live-streaming/template.html to understand it and make updates to it in Lambda etc.

Thanks

[morjoan]

For MediaPackageSettings, the ChannelId should be the Id of the MediaPackage channel you created -- in your example that is "MediaPackage2021", not MediaPackageChannel. Using REF won't give you the Id but instead the MediaPackage ARN (see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-mediapackage-channel.html).

At any rate, you'll find the source code for that template here: https://docs.aws.amazon.com/solutions/latest/live-streaming/source-code.html

Good luck!

[samueleastdev]

Hi @morjoan

Thanks I have tried that before pretty much tried every combination I could possible think of changing the id just gives this error.

output_groups[0].output_group_settings.destination.channel_id instance.destination additionalProperty "channel_id" exists in instance when not allowed; output_groups[0].output_group_settings.destination.uri instance.destination requires property "uri"; output_groups[0].output_group_settings Object does not match discriminator "hls_group" (Service: AWSMediaLive; Status Code: 422; Error Code: UnprocessableEntityException; Request ID: 3911a8f5-4e79-4fdb-b165-998a69253b25; Proxy: null)

"Destinations": [{
                    "Id": "destination1",
                    "MediaPackageSettings": [{
                        "ChannelId" : "MediaPackage2021"
                    }]
                }],

I have tried removing the Id.

"Destinations": [{
                    "MediaPackageSettings": [{
                        "ChannelId" : "MediaPackage2021"
                    }]
                }],

And it says its required.

Destination id can not be empty. (Service: AWSMediaLive; Status Code: 400; Error Code: BadRequestException; Request ID: 6c3bebe6-8bc9-444b-b6e3-94046f11a580; Proxy: null)

Even tho it says it is not required in the documentation.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-medialive-channel-outputdestination.html

Id
The ID for this destination.

Required: No

Type: String

Update requires: No interruption

Thought it maybe related to my EncoderSettings

"Destination": {
                                    "DestinationRefId": "destination1"
                                },

In my hls OutputGroups no luck.

You would think that adding the "ChannelId" : "MediaPackage2021" would be the right way to go unfortunately not.

Thanks

[richardcornelissen]

@samueleastdev ran into the same error message. For me, the issue wasn't in the Destinations config, but in the EncoderSettings. I've used HlsGroupSettings and HlsOutputSettings in the OutputGroups like this:

{
  "ChannelClass": "SINGLE_PIPELINE",
  "Destinations": [
    {
      "Id": "some-id",
      "MediaPackageSettings": [
        {
          "ChannelId": "media-package-channel-id"
        }
      ]
    }
  ],
  "EncoderSettings": {
    "OutputGroups": [
      {
        "Name": "some-name",
        "OutputGroupSettings": {
          "HlsGroupSettings": {
            ...
          }
        },
        "Outputs": [
          {
            "OutputName": "_some-name",
            "OutputSettings": {
              "HlsOutputSettings": {
                ...
              }
            }
          }
        ]
      }
    ]
  },
  "InputAttachments": [...],
  "InputSpecification": {...},
  "Name": "some-name",
  "RoleArn": "some-role",
  "Tags": {}
}

While I should've used MediaPackageGroupSettings and MediaPackageOutputSettings like so

{
  "ChannelClass": "SINGLE_PIPELINE",
  "Destinations": [
    {
      "Id": "some-id",
      "MediaPackageSettings": [
        {
          "ChannelId": "media-package-channel-id"
        }
      ]
    }
  ],
  "EncoderSettings": {
    "OutputGroups": [
      {
        "Name": "some-name",
        "OutputGroupSettings": {
          "MediaPackageGroupSettings": {
            "Destination": {
                "DestinationRefId": "media-package-channel-id"
            }
        }
        },
        "Outputs": [
          {
            "OutputName": "_some-name",
            "OutputSettings": {
              "MediaPackageOutputSettings": {}
            }
          }
        ]
      }
    ]
  },
  "InputAttachments": [...],
  "InputSpecification": {...},
  "Name": "some-name",
  "RoleArn": "some-role",
  "Tags": {}
}

[morjoan]

Hi, In case you're still running into problems, wanted to share a blog I recently published that provides a working simple CFN template using the MediaPackage resource: https://aws.amazon.com/blogs/media/sbpdx-part-2-anatomy-live-streaming-aws-cloudformation-template/

[samueleastdev]

Hi @morjoan & @richardcornelissen

Thanks for the extra information I can now get somewhere with the following template I am trying to use RTMP push.

{
    "Resources": {
        "MediaPackageChannel": {
            "Type": "AWS::MediaPackage::Channel",
            "Description": "MediaPackage Channel",
            "Properties": {
                "Id": {
                    "Fn::Sub": "${AWS::StackName}-Channel"
                }
            }
        },
        "MediaPackageEndpoint": {
            "Type": "AWS::MediaPackage::OriginEndpoint",
            "Properties": {
                "Description": "MediaPackage HLS Endpoint",
                "ChannelId": {
                    "Fn::Sub": "${AWS::StackName}-Channel"
                },
                "Id": "HLSEndpoint",
                "HlsPackage": {},
                "Origination": "ALLOW"
            },
            "DependsOn": "MediaPackageChannel"
        },
        "MediaLiveSecurityGroup": {
            "Type": "AWS::MediaLive::InputSecurityGroup",
            "Properties": {
                "WhitelistRules": [{
                  "Cidr" : "0.0.0.0/0"
                }]
            }
        },
        "MediaLiveInput": {
            "Type": "AWS::MediaLive::Input",
            "Properties": {
                "Name": {
                    "Fn::Sub": "${AWS::StackName}-RTMPInput"
                },
                "Destinations": [{
                    "StreamName" : "app/key"
                }],
                "Type": "RTMP_PUSH",
                "InputSecurityGroups": [{
                    "Ref": "MediaLiveSecurityGroup"
                }]
            }
        },
        "SSMParameter": {
            "Type": "AWS::SSM::Parameter",
            "Properties": {
                "Description": "Automatically created by AWS Elemental",
                "Name": "my_destination1",
                "Type": "String",
                "Value": "my_destination1"
            }
        },
        "MediaLiveChannel": {
            "Type": "AWS::MediaLive::Channel",
            "Properties": {
                "Name": {
                    "Fn::Sub": "${AWS::StackName}-Channel"
                },
                "ChannelClass": "SINGLE_PIPELINE",
                "RoleArn": {
                    "Ref": "MediaLiveRoleArn"
                },
                "EncoderSettings": {
                    "AudioDescriptions": [
                        {
                            "Name": "audio_1",
                            "LanguageCodeControl": "FOLLOW_INPUT",
                            "AudioTypeControl": "FOLLOW_INPUT"
                        }
                    ],
                    "OutputGroups": [
                        {
                            "OutputGroupSettings": {
                                "MediaPackageGroupSettings": {
                                    "Destination": {
                                        "DestinationRefId": "destination1"
                                    }
                                }
                            },
                            "Outputs": [
                                {
                                    "AudioDescriptionNames": [
                                        "audio_1"
                                    ],
                                    "CaptionDescriptionNames": [],
                                    "OutputName": "EMPOutput1",
                                    "OutputSettings": {
                                        "MediaPackageOutputSettings": {}
                                    },
                                    "VideoDescriptionName": "video_1"
                                }
                            ]
                        }
                    ],
                    "VideoDescriptions": [
                        {
                            "CodecSettings": {
                                "H264Settings": {
                                    "AdaptiveQuantization": "MEDIUM",
                                    "AfdSignaling": "NONE",
                                    "ColorMetadata": "INSERT",
                                    "EntropyEncoding": "CABAC",
                                    "FlickerAq": "ENABLED",
                                    "ForceFieldPictures": "DISABLED",
                                    "FramerateControl": "SPECIFIED",
                                    "FramerateDenominator": 1,
                                    "FramerateNumerator": 30,
                                    "GopBReference": "DISABLED",
                                    "GopClosedCadence": 1,
                                    "GopNumBFrames": 2,
                                    "GopSize": 90,
                                    "GopSizeUnits": "FRAMES",
                                    "Level": "H264_LEVEL_AUTO",
                                    "LookAheadRateControl": "MEDIUM",
                                    "NumRefFrames": 1,
                                    "ParControl": "SPECIFIED",
                                    "ParDenominator": 1,
                                    "ParNumerator": 1,
                                    "Profile": "MAIN",
                                    "RateControlMode": "CBR",
                                    "ScanType": "PROGRESSIVE",
                                    "SceneChangeDetect": "ENABLED",
                                    "SpatialAq": "ENABLED",
                                    "SubgopLength": "FIXED",
                                    "Syntax": "DEFAULT",
                                    "TemporalAq": "ENABLED",
                                    "TimecodeInsertion": "DISABLED"
                                }
                            },
                            "Height": 720,
                            "Name": "video_1",
                            "RespondToAfd": "NONE",
                            "ScalingBehavior": "DEFAULT",
                            "Sharpness": 50,
                            "Width": 1280
                        }
                    ],
                    "TimecodeConfig": {
                        "Source": "EMBEDDED"
                    }
                },
                "Destinations": [
                    {
                        "Id": "destination1",
                        "MediaPackageSettings": [
                            {
                                "ChannelId": {
                                    "Fn::Sub": "${AWS::StackName}-Channel"
                                }
                            }
                        ]
                    }
                ],
                "InputAttachments": [{
                    "InputAttachmentName": "HLSInput",
                    "InputId": {
                        "Ref": "MediaLiveInput"
                    },
                    "InputSettings": {
                        "NetworkInputSettings": {
                            "ServerValidation": "CHECK_CRYPTOGRAPHY_AND_VALIDATE_NAME"
                        },
                        "SourceEndBehavior": "CONTINUE",
                        "InputFilter": "AUTO",
                        "FilterStrength": 1,
                        "DeblockFilter": "DISABLED",
                        "DenoiseFilter": "DISABLED",
                        "AudioSelectors": [],
                        "CaptionSelectors": [{
                            "SelectorSettings": {
                                "EmbeddedSourceSettings": {
                                    "Source608ChannelNumber": 1,
                                    "Source608TrackNumber": 1,
                                    "Convert608To708": "DISABLED",
                                    "Scte20Detection": "OFF"
                                }
                            },
                            "Name": "EmbeddedSelector"
                        }]
                    } 
                }]
            }
        }
    },
    "Parameters": {
        "MediaLiveRoleArn": {
            "Description": "ARN of the IAM role that MediaLive is allowed to assume in order to perform the operations on the resources specified in the policies.",
            "Type": "String"
        }
    },
    "Outputs": {
        "LiveStreamUrl": {
            "Description": "LiveStream Playback URL",
            "Value": {
                "Fn::Sub": "${MediaPackageEndpoint.Url}"
            }
        }
    }
}

It completes but there seems to be errors pushing to the MediaPackage channel hls endpoint that I cannot figure out.

I get the following errors.

OutputDataBackground failed to send file for URL [mediapackage://lopsss-Channel/channel_EMPOutput1_20210513T123222_00257.ts], after [3] attempts, error [failed to write output to mediapackage]

Failed to Create Output File or Socket

Failed to Write to Output

Failed to Write to Output

Failed to Write to Output

So it must be some permissions issue.

When using OBS software it makes the connection to the input.

Server: rtmp://54.000.000.243:1935/app Stream Key: key

Any suggestions on where I might be going wrong?