dotnet / arcade

Tools that provide common build infrastructure for multiple .NET Foundation projects.
MIT License
672 stars 347 forks source link

Improve copy signed binaries to drop in release pipeline #6128

Closed chcosta closed 3 years ago

chcosta commented 4 years ago

The signing stage of the release pipeline copies binaries back to the source drop. Copying back to the share takes a lot (over an hour) of time. We could improve this copy time by not copying everything back to the share, just copying the ItemsToSign that were signed.

https://dev.azure.com/dnceng/internal/_build/results?buildId=802298&view=logs&j=9ff0e0af-24e7-5b0f-fa76-5e12fc9920ef&t=4370ca2f-a014-5f8d-3be0-811b20e78569

FYI @mmitche @jcagme

jcagme commented 4 years ago

For this though, the easiest would be to keep the same folder structure of that in the drop so we just replace unsigned bits with signed and we avoid looking and replacing, but yeah, totally doable

chcosta commented 4 years ago

For this though, the easiest would be to keep the same folder structure of that in the drop so we just replace unsigned bits with signed and we avoid looking and replacing, but yeah, totally doable

Yes, we should keep the same folder structure. I'm not really sure what you're stating.

jcagme commented 4 years ago

For this though, the easiest would be to keep the same folder structure of that in the drop so we just replace unsigned bits with signed and we avoid looking and replacing, but yeah, totally doable

Yes, we should keep the same folder structure. I'm not really sure what you're stating.

Today we download all the shipping and nonshipping folder, sign the things that need to be signed in there and re-upload the whole directory. If instead, we only want to upload things that were signed, we should maintain the same drop like structure of those assuming we will move the signable files to a different folder. We can talk implementation offline if needed

chcosta commented 4 years ago

I just added "SkipUnchangedFiles" to the scripts (silly that I forgot about that property). Hopefully that improves copy times!

chcosta commented 4 years ago

Triage: let's prototype some things. maybe robocopy?

missymessa commented 3 years ago

This work has been completed. Closing.