core: Over-aggressive cache fingerprinting on synthesize

[JGillam]

Describe the bug

It appears that cdk synth triggers a call to core/lib/fs/fingerprint.js, specifically the function contentFingerprintMiss(file). This includes files that would normally be ignored, such as those in .git or .idea (or whatever IDE project folder you use).

This call can cause problems when some of these files are locked. For example, .idea now contains a lock file if the CoPilot plugin is enabled, meaning a cdk synth run on any project open in a current version of any Windows install of IntelliJ IDE (e.g. Webstorm) with the CoPilot extension may now fail, with a resulting error message starting with:

Error: EBUSY: resource busy or locked, read

and ending with

{      
 errno: -4082,
 code: 'EBUSY',
 syscall: 'read'
}

I don't know if it is relevant but this particular project I discovered this behavior on was also a mono-repo using pnpm.

Expected Behavior

The cdk synth command should not fail when a file path that is not relevant to the cdk stack contains a locked file.

Current Behavior

A file path containing a locked file in the same project as the cdk may cause the a cdk synth command to fail.

Reproduction Steps

Unfortunately I don't have a code snippet to reproduce this and my project is private so I can't share it. Building a simple cdk project does not appear sufficient to trigger the issue but my complex pnpm monorepo project reproduces it consistently.

Possible Solution

I would suggest a configuration (in cdk.json or in a .cdkignore file) that controls the scope of the cdk cache/fingerprint better. It should not be looking in .git or .idea (and probably other places) anyway. Similar to a .gitignore file, there should be a way to tell cdk synth which paths should be excluded entirely.

Additional Information/Context

When troubleshooting this I added a log message to core/lib/fs/fingerprint.js as follows:

function contentFingerprintMiss(file) {
  console.log(`Fingerprinting file: ${file}`);

Then when I do a cdk synth I can see a list of all the files getting called with a fingerprint miss. That list includes many files under .git and .idea (I use WebStorm), that cdk synth should not even care about. See the output snippet below where it hits the xd.lck file that is part of the copilot extension:

...
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\00000000000.xd
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\blobs\version
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\xd.lck
Error: EBUSY: resource busy or locked, read
   at Object.readSync (node:fs:752:18)
   at contentFingerprintMiss (C:\Users\JasonGillam\Projects\my-project\node_modules\.pnpm\aws-cdk-lib@2.121.1_constructs@10.3.0\node_modules\aws-cdk-lib\core\lib\fs\fingerprint.js:78:25)
...

I tested disabling the copilot extension, and this also resolves the issue, since it removes the lock file.

CDK CLI Version

2.131.0

Framework Version

No response

Node.js Version

20.11.1

OS

Windows 11

Language

TypeScript

Language Version

"typescript": "~5.3.3"

Other information

No response

[pahud]

Thanks for the report. We'll look into this issue and discuss with the team.

[enkhee]

.idea/copilot/chatSessions/xd.lck [error] Unable to read file ".idea/copilot/chatSessions/xd.lck": [error] EBUSY: resource busy or locked, read

[JGillam]

The immediate issue this was causing for me has been addressed in version 1.5.1.5285 of the CoPilot plugin for IntelliJ. That chatSessions lock file is now stored elsewhere. However, I still believe the core issue I reported should be addressed.

[systematicguy]

I can reproduce this quite easily, having a solution in Visual Studio open. For us, the blocker is the .vs/redacted/FileContextIndex/blabla.vsidx. Our structure:

ProjectRoot/
  .vs/
  CdkInfrastructure/
  Lambda1Src/
  Lambda2Src/
  App.sln

[rix0rrr]

Can you provide a complete app that displays this behavior? Normally we'd fingerprint only assets, and the .git and .idea directories would be in your project root, not an asset directory.

Also, usually you can specify ignore directories when you specify assets. That's why a reproducing application would help.