Working Azure DevOps Pipeline Caching for Angular CI | josh-ops

[utterances-bot]

Working Azure DevOps Pipeline Caching for Angular CI | josh-ops

I share how I finally got the Pipeline Cache task to work with my Angular build pipeline

https://josh-ops.com/posts/azdo-angular-pipeline-caching/

[Mitesh1711]

Hi Josh, Thanks for the answer, can you please post the npm install task script. Want to make sure that I am on correct path.

[joshjohanning]

@Mitesh1711 certainly! I should have had included that in the original post, good idea :)

The npm install task for this example would be:

  - task: Npm@1
    displayName: npm install
    inputs:
      command: install
      workingDir: $(Build.SourcesDirectory)/Source/MyWeb

The workindDir is the directory where the package.json file exists.

[awsservices4]

hi The cache task is working and getting optimised but when deploying the latest changes are not effecting in Live.. environment. what might be the issue? of not reflecting the latest changes

[awsservices4]

we are using microsoft hosted agents and deploying into it..and will this cache task works in microsift hosted agents?

[joshjohanning]

@awsservices4 it should totally work for Hosted Agents!

when deploying the latest changes are not effecting in Live.. environment. what might be the issue? of not reflecting the latest changes

What do you mean, like where you deploy to is still showing an old version of your site?

This cache task is primarily used to cache packages (like npm packages) to speed up build time, so it shouldn't affect your build output / deployment to your environment.

[awsservices4]

Thanks for the reply. Yes cache is working fine.then after deploying only the changes are not effecting ie the old changes remain same even after changing .

In the same pipeline if i am not using Cache task then it's working properly Can you tell me where might be the problm

Thanks

On Fri, 30 Sep, 2022, 10:35 pm Josh Johanning, @.***> wrote:

@awsservices4 https://github.com/awsservices4 it should totally work for Hosted Agents!

when deploying the latest changes are not effecting in Live.. environment. what might be the issue? of not reflecting the latest changes

What do you mean, like where you deploy to is still showing an old version of your site?

This cache task is primarily used to cache packages (like npm packages) to speed up build time, so it shouldn't affect your build output / deployment to your environment.

— Reply to this email directly, view it on GitHub https://github.com/joshjohanning/joshjohanning.github.io/issues/7#issuecomment-1263812922, or unsubscribe https://github.com/notifications/unsubscribe-auth/AREI6CQ7X3ZKKIIXZUA637DWA4MV5ANCNFSM5IISYJ6A . You are receiving this because you were mentioned.Message ID: @.***>

[joshjohanning]

@awsservices4 oh that's really weird. Is your key (cache hit) setting off? This is what value it's going to use to determine whether to use the cache or not. Maybe you need to switch to/from your package.json to package-lock.json, or similar.

It should only affect what values you have stored in the node_modules folder (if that's the folder you're caching), it should still run an npm restore and npm build every time which should re-build and package your output to publish?

I would also be interested what's not being updated on the site - is it related to package versions (ie: old dependency rendering calendar wrong), or other views/files that are being changed that aren't being reflected?

[lzandman]

Unfortunately I fear you've managed to succeed in the fine art of Failing Succesfully ;-D

The problem is you're using npm install in a CI/CD pipeline. However, that's not the way! You should use npm ci, which is what the example code for the Cachine Task uses. I refer you to the NPM docs/Google for all differences between npm install and npm ci, but summarized: a pipeline needs to be dependable and predictable and that what npm ci in combination with a package-lock.json file ensures. One of the first things npm ci does is actually delete the node_modules folder. So caching that folder doesn't really make sense in this scenario.

The Caching Task docs refer to the creation of a pipeline cache of the NPM cache folder, which isn't the same as the node_modules folder! Basically the NPM cache contains the downloaded NPM packages in their orginal compressed form, while the node_modules folder in generated from that cache and contains packages in their uncompressed form (and therefore it's typically way larger in size and contains many more files).

The Caching Task docs assume NPM creates this cache folder at the $(Pipeline.Workspace)/.npm location. Depending on your platform this might not be the case. So you should actually specify this location yourself during npm ci by using the --cache command line argument. So something like this:

variables:
  npm_config_cache: $(Pipeline.Workspace)/.npm

steps:
- task: Cache@2
  inputs:
    key: 'npm | "$(Agent.OS)" | package-lock.json'
    restoreKeys: |
       npm | "$(Agent.OS)"
    path: $(npm_config_cache)
  displayName: Cache npm

- script: npm ci --cache $(npm_config_cache)

So, while your solution probably works, you're basically doing desktop development stuff in a CI/CD pipeline. Which, again, isn't the way!

Hope this helps!

[joshjohanning]

@lzandman TIL, thank you for educating me! I appreciate the example here too ❤️ .

Isn't it disappointing then that the default Azure DevOps pipeline template for each Node example use npm install ? :

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '10.x'
  displayName: 'Install Node.js'

- script: |
    npm install -g @angular/cli
    npm install
    ng build --prod
  displayName: 'npm install and build'

I will admit, I like how GitHub Actions does the caching a little better, as it's handled pretty much automatically in the setup-* actions (e.g.: setup-node).

Thanks again!

[fenilshahdev]

If I am using condition on NPM Install, then task is still executing. I verified with the restore keys as well and that all looks good. Could you please help me with the script for caching and conditional based installation of npm install? Thank you

[joshjohanning]

If I am using condition on NPM Install, then task is still executing. I verified with the restore keys as well and that all looks good. Could you please help me with the script for caching and conditional based installation of npm install? Thank you

@fenilshahdev I think you would want something like this, using the cacheHitVar and a condition on the npm install step.

steps:
- task: Cache@2
  displayName: load npm cache
  inputs:
    key: npm | $(Agent.OS) | $(Build.SourcesDirectory)/Source/MyWeb/package.json
    restoreKeys: |
        npm | "$(Agent.OS)"
    path: $(Build.SourcesDirectory)/Source/MyWeb/node_modules
    cacheHitVar: CACHE_HIT

- name: npm install
  condition: ne(variables.CACHE_HIT, 'true')
  script: | 
    npm install

[Gopinath1906]

Hi Josh-ops, I am trying to implement cache in az pipeline as node project. Cache path is agent mount path (/adoagent-cache/) ,We are doing delete the current pipeline workspace. I am getting this message always "there is cache miss"

It supposed to be created under /adoagent-cache/.npm/ but It was created under “/adoagent-cache/.npm/_npx/73b02210abc194ff”.They should be created under cache path “/adoagent-cache/.npm/”. Additionally, I checked and found a mismatch between the local repositories.

Could you please help on this issue.

Cache Script:

- task: Cache@2
  inputs:
    key: 'npm | "$(Agent.OS)" |$(Build.Repository.Name)/${{parameters.npmFilePath}}/package-lock.json'
    restoreKeys: |
      npm | "$(Agent.OS)"
    path: $(Pipeline.Workspace)/.npm
    cacheHitVar: 'CacheRestored'
  displayName: load Cache Npm

Delete Files:

      - task: DeleteFiles@1
        displayName: 'Cleaning up Pipeline Workspace'
        condition: always()
        inputs:
          SourceFolder: '$(Pipeline.Workspace)'
          Contents: '*/**'
          RemoveDotFiles: true