search

Azure / azure-media-migration

Azure Media Migration Tool
https://github.com/Azure/azure-media-migration
MIT License
11 stars 15 forks source link

Parameters throws null ref exception when trying to migrate AMS video that has a secondary audio in it #220

Closed srayan closed 10 months ago

srayan commented 10 months ago

This is happening because in ShakaPackager.cs Ln 131 we have var source = t.Parameters.SingleOrDefault(p => p.Name == TRANSCRIPT_SOURCE)?.Value ?? t.Source;

When trying to migrate a video from AMS to a storage account, if the video has a secondary audio in it, then there are tracks that are being sent to this line. 1 video track and 2 audio tracks. While the audio tracks have Parameters populated, the video track does not.

So, the solution should be to add null handling here, thus change FROM : var source = t.Parameters.SingleOrDefault(p => p.Name == TRANSCRIPT_SOURCE)?.Value ?? t.Source; TO: var source = t.Parameters?.SingleOrDefault(p => p.Name == TRANSCRIPT_SOURCE)?.Value ?? t.Source;

I tried to send a PR with this fix, but I keep getting a 403. Please advise.

Stack trace - 2023-11-15 14:29:05.598 -08:00 [INF] Begin migration of assets for account: ampinfrawestuscivideo 2023-11-15 14:29:11.714 -08:00 [INF] The total asset count of the media account is 0. 2023-11-15 14:29:19.417 -08:00 [INF] The total assets to handle in this run is 2. 2023-11-15 14:29:28.660 -08:00 [INF] Migrating asset: amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5 (container asset-7757f8d7-7671-40f9-9d72-89fe04c56d14) ... 2023-11-15 14:29:28.660 -08:00 [INF] Migrating asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 (container asset-a330595d-1ef7-4cf7-bfbf-6951f9ce49c0) ... 2023-11-15 14:29:29.651 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a.ism 2023-11-15 14:29:29.651 -08:00 [INF] Asset: amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5 has 1 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a31e3.mp4 2023-11-15 14:29:32.534 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a_650.mp4 2023-11-15 14:29:32.636 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a_650_1.mpi 2023-11-15 14:29:32.735 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a_650_2.mpi 2023-11-15 14:29:32.829 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: 27dcfdca-a90b-4c27-9481-b093352a_manifest.json 2023-11-15 14:29:32.902 -08:00 [INF] Streaming locator name and locator Id is: f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810 2023-11-15 14:29:32.952 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: e39e593a-0d00-8176-b850-f1ee83f44339_metadata.json 2023-11-15 14:29:33.075 -08:00 [INF] Asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 has 7 blobs and content: en-usa9f283b0-645e-48ab-a524-a40d6a79.mp4 2023-11-15 14:29:34.108 -08:00 [INF] Streaming locator name and locator Id is: fa548f04-0e41-40b0-ba3e-ff85ad683e8c 2023-11-15 14:29:40.622 -08:00 [WRN] No manifest (.ism file) found in container asset-7757f8d7-7671-40f9-9d72-89fe04c56d14 2023-11-15 14:29:46.436 -08:00 [VRB] Template ${ContainerName}/ expanded to f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810/ 2023-11-15 14:29:46.941 -08:00 [VRB] Unknown attribute in manifest title 2023-11-15 14:29:46.944 -08:00 [VRB] Found manifest 27dcfdca-a90b-4c27-9481-b093352a.ism of format mp4 in container asset-a330595d-1ef7-4cf7-bfbf-6951f9ce49c0 2023-11-15 14:29:48.624 -08:00 [VRB] Template ${ContainerName}/ expanded to fa548f04-0e41-40b0-ba3e-ff85ad683e8c/ 2023-11-15 14:29:54.397 -08:00 [VRB] Template ${ContainerName}/ expanded to f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810/ 2023-11-15 14:29:54.398 -08:00 [VRB] Asset amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5 is in format: null. 2023-11-15 14:29:54.398 -08:00 [INF] The asset amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5 with format non_ism is not supported by transform PackageTransform in current version, try next transform... 2023-11-15 14:29:54.401 -08:00 [VRB] Template ${ContainerName}/ expanded to f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810/ 2023-11-15 14:29:54.401 -08:00 [VRB] Asset amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5 is in format: null. 2023-11-15 14:29:54.640 -08:00 [VRB] Uploading to /27dcfdca-a90b-4c27-9481-b093352a31e3.mp4 in container f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810 of account: ampwusdevadp002... 2023-11-15 14:29:55.884 -08:00 [VRB] Template ${ContainerName}/ expanded to fa548f04-0e41-40b0-ba3e-ff85ad683e8c/ 2023-11-15 14:29:55.884 -08:00 [VRB] Asset amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 is in format: mp4. 2023-11-15 14:30:00.851 -08:00 [VRB] Uploading to /27dcfdca-a90b-4c27-9481-b093352a_manifest.json in container fa548f04-0e41-40b0-ba3e-ff85ad683e8c of account: ampwusdevadp002... 2023-11-15 14:30:00.984 -08:00 [VRB] Uploading to /e39e593a-0d00-8176-b850-f1ee83f44339_metadata.json in container fa548f04-0e41-40b0-ba3e-ff85ad683e8c of account: ampwusdevadp002... 2023-11-15 14:30:01.004 -08:00 [DBG] Begin downloading 27dcfdca-a90b-4c27-9481-b093352a_650.mp4 2023-11-15 14:30:01.147 -08:00 [DBG] Begin downloading en-usa9f283b0-645e-48ab-a524-a40d6a79.mp4 2023-11-15 14:30:01.404 -08:00 [VRB] Uploaded 0 bytes to /27dcfdca-a90b-4c27-9481-b093352a31e3.mp4 2023-11-15 14:30:01.448 -08:00 [DBG] Finished download of en-usa9f283b0-645e-48ab-a524-a40d6a79.mp4 2023-11-15 14:30:02.141 -08:00 [DBG] Finished download of 27dcfdca-a90b-4c27-9481-b093352a_650.mp4 2023-11-15 14:30:02.460 -08:00 [VRB] The lease-detect blob for output path / under f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810 is deleted. 2023-11-15 14:30:02.517 -08:00 [VRB] Starting static packaging for asset amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769... 2023-11-15 14:30:09.285 -08:00 [DBG] Migrated asset: amp:AA1uLYK:1001:9b94d366-1c49-4a2f-86af-37bb3f385fe5, container: f5efa6d4-6efa-4eb7-ba65-dd3ba3da3810, type: non_ism, status: "Completed" 2023-11-15 14:30:09.345 -08:00 [DBG] Asset Migration: 1/2 Assets 2023-11-15 14:30:09.363 -08:00 [ERR] Failed to migrate asset amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769 Error:System.ArgumentNullException: Value cannot be null. (Parameter 'source') at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument) at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable1 source, Func2 predicate, Boolean& found) at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable1 source, Func2 predicate) at AMSMigrate.Transform.ShakaPackager.<>c__DisplayClass6_0.b__0(Track t, Int32 i) in C:\git\CMS.Service.Video.Processor\Src\azure-media-migration\migrationTool\transform\ShakaPackager.cs:line 131 at System.Linq.Enumerable.SelectIterator[TSource,TResult](IEnumerable1 source, Func3 selector)+MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at AMSMigrate.Transform.ShakaPackager.GetArguments(IList1 inputs, IList1 outputs, IList1 manifests) in C:\git\CMS.Service.Video.Processor\Src\azure-media-migration\migrationTool\transform\ShakaPackager.cs:line 127 at AMSMigrate.Transform.ShakaPackager.RunAsync(String workingDirectory, String[] inputs, String[] outputs, String[] manifests, CancellationToken cancellationToken) in C:\git\CMS.Service.Video.Processor\Src\azure-media-migration\migrationTool\transform\ShakaPackager.cs:line 297 at AMSMigrate.Transform.PackageTransform.TransformAsync(AssetDetails details, ValueTuple2 outputPath, CancellationToken cancellationToken) in C:\git\CMS.Service.Video.Processor\Src\azure-media-migration\migrationTool\transform\PackageTransform.cs:line 161 2023-11-15 14:30:10.602 -08:00 [VRB] Uploaded 0 bytes to /27dcfdca-a90b-4c27-9481-b093352a_manifest.json 2023-11-15 14:30:10.603 -08:00 [VRB] Uploaded 1013 bytes to /27dcfdca-a90b-4c27-9481-b093352a_manifest.json 2023-11-15 14:30:12.655 -08:00 [VRB] Uploaded 0 bytes to /e39e593a-0d00-8176-b850-f1ee83f44339_metadata.json 2023-11-15 14:30:12.655 -08:00 [VRB] Uploaded 3698 bytes to /e39e593a-0d00-8176-b850-f1ee83f44339_metadata.json 2023-11-15 14:30:12.873 -08:00 [VRB] The lease-detect blob for output path / under fa548f04-0e41-40b0-ba3e-ff85ad683e8c is deleted. 2023-11-15 14:30:12.995 -08:00 [DBG] Migrated asset: amp:AA1uLYK:1004:1a5f125b-29e6-4c58-9e5f-622410714769, container: fa548f04-0e41-40b0-ba3e-ff85ad683e8c, type: mp4, status: "Failed" 2023-11-15 14:30:12.999 -08:00 [DBG] Asset Migration: 2/2 Assets 2023-11-15 14:30:13.002 -08:00 [INF] Finished migration of assets for account: ampinfrawestuscivideo. Time taken: "00:01:25.8490202" 2023-11-15 14:30:13.024 -08:00 [INF] See file C:\git\CMS.Service.Video.Processor\Src\azure-media-migration\migrationTool\bin\Debug\net6.0\MigrationLog_14_28_44.txt for detailed logs.

pohhsu commented 10 months ago

fixed by this commit

75b20dd When migrating a video with a secondary audio component in it, the video track has Parameters set to null. It was throwing an exception here sguhat@microsoft.com