search

Azure / azure-media-migration

Azure Media Migration Tool
https://github.com/Azure/azure-media-migration
MIT License
10 stars 13 forks source link

Add option to specify a file that contains all asset guids that should be migrated #247

Closed schuettecarsten closed 1 month ago

schuettecarsten commented 1 month ago

This PR adds an option to the "assets" command that allows to specify a text file containing a list of asset GUIDs that should be migrated. I also updated the maximum value of batchSize to 20 to utilize a large size machine better.

duggaraju commented 1 month ago

Any reason why it has to be in a text file and you can't use the --resource-filter arguments to pass the asset ids on the comman dline? This is only meant for testing right. So you are not using it to target a large list of asset ids

schuettecarsten commented 1 month ago

We have >10.000 assets in the AMS on one of our customers but only want migrate about 3.500 of them. So we export the asset IDs from our database into a TXT file and then call the migration tool.

duggaraju commented 1 month ago

I am weary of taking this fix as a generic solution for everyone for the following reasons.

  1. This options conflicts with the --resource-filter option and we should only allow one or other.
  2. All the filtering is on client side and you will still load all assets in memory and filter. What if there were 100,000 assets and you only want 10. Then this is very inefficient.

If you already have a list of ids in your database, then you can query them in your wrapper application/library and call the migration tool by passing them on command line (once or multiple times depending on how many asset ids you have) Of course you are free to make these changes in your fork.

schuettecarsten commented 1 month ago

I am completely fine with that :-)