microsoft / azure-pipelines-tool-lib

Lib for creating CI/CD tool installer tasks in Azure Pipelines
MIT License
81 stars 57 forks source link

[PERF] Extract directly to cache location #67

Open jessehouwing opened 4 years ago

jessehouwing commented 4 years ago

This is probably a architectural choice to not accidentally corrupt the cache directory, but the additional copy step adds a significant amount of time to each build (especially on the hosted agents).

The process one should follow:

This wastes about 34 seconds when using the UseNode task for example:

2020-04-23T08:41:16.3375719Z Caching tool: node 10.20.1 x64
2020-04-23T08:41:50.2986887Z Prepending PATH environment variable with directory: C:\hostedtoolcache\windows\node\10.20.1\x64

And on the Hosted agent it makes no sense at all, since the image is destroyed after the build completes. There is zero benefit, apart from consistency, in populating the _tools folder.

Options:

  1. Detect this is a throw-away image and skip the copy step, adding the path directly to the temp location.
  2. Extract directly into the _tools directory, possibly use marker files to signal success.
PAULNICEITISWHATITIS commented 4 months ago

This is probably a architectural choice to not accidentally corrupt the cache directory, but the additional copy step adds a significant amount of time to each build (especially on the hosted agents).

The process one should follow:

  • Download payload
  • Extract to temp location
  • Use Tool Lib to cache the bits

    • copies all contents from temp to the _tools folder on the agent.

This wastes about 34 seconds when using the UseNode task for example:

2020-04-23T08:41:16.3375719Z Caching tool: node 10.20.1 x64
2020-04-23T08:41:50.2986887Z Prepending PATH environment variable with directory: C:\hostedtoolcache\windows\node\10.20.1\x64

And on the Hosted agent it makes no sense at all, since the image is destroyed after the build completes. There is zero benefit, apart from consistency, in populating the _tools folder.

Options:

  1. Detect this is a throw-away image and skip the copy step, adding the path directly to the temp location.
  2. Extract directly into the _tools directory, possibly use marker files to signal success.
PAULNICEITISWHATITIS commented 4 months ago

218 #227 #224

WIN_20240229_09_09_30_Pro #