remy / nodemon

Monitor for any changes in your node.js application and automatically restart the server - perfect for development
http://nodemon.io/
MIT License
26.23k stars 1.72k forks source link

Files are watched even if in the ignore list #2057

Closed cberescu closed 2 years ago

cberescu commented 2 years ago

RUN npm install -g nodemon

RUN mkdir -p /app WORKDIR /app

CMD ["nodemon", "--exec", "npm start || touch _keepalive.js"]

- Command you ran: `npm install && node app.js`

### Expected behaviour
should ignore all files that are set in the nodemon config 

{ "verbose": true, "ignore": ["package-lock.json", "node_modules/","logs/","cache/*"], "delay": 3000 }


### Actual behaviour
It still parses the directories. The cache directory has around 7mil files, if i remove the directory outside the node app everything works as expected, as soon as i move it back i get : `[nodemon] Internal watch failed: ENOSPC: System limit for number of file watchers reached, watch '/app/cache/288_162/6963394_288_162.jpg'`

What i think happens , all files are watched, even in the ignore directory,  and if they are changed nodemon does not restart the app 
but it knows that the file was changed. 

I think it should not watch files that are in the ignore list. 

### Steps to reproduce
I guess create a directory with 7mil files ? 

---

[nodemon] 2.0.19 [nodemon] reading config ./nodemon.json [nodemon] to restart at any time, enter rs [nodemon] or send SIGHUP to 1 to restart [nodemon] ignoring: package-lock.json node_modules//* logs/*/ cache// [nodemon] watching path(s): .* [nodemon] watching extensions: js,mjs,json [nodemon] starting npm start || touch _keepalive.js [nodemon] spawning [nodemon] child pid: 26 [nodemon] watching 15 files



If applicable, please append the `--dump` flag on your command and include the output here **ensuring to remove any sensitive/personal details or tokens**.
remy commented 2 years ago

To know what to watch and what to ignore (for restart) it still needs to enumerate all the files in the tree.

Don't run nodemon from a root system folder, or rather run it in the project scope.

On Mon, 15 Aug 2022, 18:31 cberescu, @.***> wrote:

  • nodemon -v: 2.0.19
  • node -v: v16.16.0
  • Operating system/terminal environment: docker
  • Using Docker? What image: Custom

FROM node:16.16.0 COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"]

RUN npm install -g nodemon

RUN mkdir -p /app WORKDIR /app

CMD ["nodemon", "--exec", "npm start || touch _keepalive.js"]

  • Command you ran: npm install && node app.js

Expected behaviour

should ignore all files that are set in the nodemon config

{ "verbose": true, "ignore": ["package-lock.json", "node_modules/","logs/","cache/*"], "delay": 3000 }

Actual behaviour

It still parses the directories. The cache directory has around 7mil files, if i remove the directory outside the node app everything works as expected, as soon as i move it back i get : [nodemon] Internal watch failed: ENOSPC: System limit for number of file watchers reached, watch '/app/cache/288_162/6963394_288_162.jpg'

What i think happens , all files are watched, even in the ignore directory, and if they are changed nodemon does not restart the app but it knows that the file was changed.

I think it should not watch files that are in the ignore list. Steps to reproduce

I guess create a directory with 7mil files ?

[nodemon] 2.0.19 [nodemon] reading config ./nodemon.json [nodemon] to restart at any time, enter rs [nodemon] or send SIGHUP to 1 to restart [nodemon] ignoring: package-lock.json node_modules//* logs/*/ cache// [nodemon] watching path(s): .* [nodemon] watching extensions: js,mjs,json [nodemon] starting npm start || touch _keepalive.js [nodemon] spawning [nodemon] child pid: 26 [nodemon] watching 15 files

If applicable, please append the --dump flag on your command and include the output here ensuring to remove any sensitive/personal details or tokens.

— Reply to this email directly, view it on GitHub https://github.com/remy/nodemon/issues/2057, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADLBEXA7N5Y3AWGJJQOXLVZJ4WTANCNFSM56S5DRGA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

cberescu commented 2 years ago

Hi @remy and thanks for the answer .

If i put in the ignore list this : cache/*/ , in what situation does it need to watch the files inside it ?

I am running it in the project scope. It is a images server in witch i choose to keep a folder with cached images inside the project scope.

remy commented 2 years ago

"watch" is misleading, nodemon needs to look at files initially to match what to watch and ignore.

If you change from the negative to additive, you can say --watch src/ which means it'll not need to look at other directories.

Failing that, run with debug env to find out and share what's happening: DEBUG=nodemon:* <CMD> it should list what's going on.

cberescu commented 2 years ago

So, i added the debug env and i get this :

2022-08-15T19:15:25.353Z nodemon:watch chokidar watching: /app/cache/160_153/5018032_160_153.jpeg
2022-08-15T19:15:25.353Z nodemon:watch chokidar watching: /app/cache/160_153/5018036_160_153.jpg
2022-08-15T19:15:25.353Z nodemon:watch chokidar watching: /app/cache/160_153/5018055_160_153.jpg
2022-08-15T19:15:25.353Z nodemon:watch chokidar watching: /app/cache/160_153/5018215_160_153.jpg
2022-08-15T19:15:25.359Z nodemon:watch chokidar watching: /app/cache/160_153/5018216_160_153.png
2022-08-15T19:15:25.359Z nodemon:watch chokidar watching: /app/cache/160_153/5018285_160_153.jpg
2022-08-15T19:15:25.359Z nodemon:watch chokidar watching: /app/cache/160_153/5018324_160_153.png
2022-08-15T19:15:25.360Z nodemon:watch chokidar watching: /app/cache/160_153/5018506_160_153.jpg
2022-08-15T19:15:25.360Z nodemon:watch chokidar watching: /app/cache/160_153/5018508_160_153.jpg
2022-08-15T19:15:25.360Z nodemon:watch chokidar watching: /app/cache/160_153/5018569_160_153.jpg
2022-08-15T19:15:25.360Z nodemon:watch chokidar watching: /app/cache/160_153/5018672_160_153.jpg
2022-08-15T19:15:25.360Z nodemon:watch chokidar watching: /app/cache/160_153/5018701_160_153.jpeg
2022-08-15T19:15:25.366Z nodemon:watch chokidar watching: /app/cache/160_153/5018778_160_153.png
2022-08-15T19:15:25.367Z nodemon:watch chokidar watching: /app/cache/160_153/5018780_160_153.jpg
2022-08-15T19:15:25.367Z nodemon:watch chokidar watching: /app/cache/160_153/5018891_160_153.jpg
2022-08-15T19:15:25.367Z nodemon:watch chokidar watching: /app/cache/160_153/5018907_160_153.jpg

witch i think it should not even enter in that folder as everything in it needs to be ignored. Also it takes .jpg/.png/.etc extension, witch aren't in the default list.

cberescu commented 2 years ago

Also, the beginning looks like this ;

2022-08-15T19:12:48.339Z nodemon:run spawn sh -c npm start || touch _keepalive.js
2022-08-15T19:12:48.341Z nodemon:run start watch on: [ '*.*', re: /.*\..*/ ]
2022-08-15T19:12:48.373Z nodemon:watch chokidar watching: /app/app.js
2022-08-15T19:12:48.375Z nodemon:watch chokidar watching: /app/config-sample.json
2022-08-15T19:12:48.375Z nodemon:watch chokidar watching: /app/config.json
2022-08-15T19:12:48.376Z nodemon:watch chokidar watching: /app/favicon.ico
2022-08-15T19:12:48.376Z nodemon:watch chokidar watching: /app/nodemon.json
2022-08-15T19:12:48.377Z nodemon:watch chokidar watching: /app/package.json
2022-08-15T19:12:48.708Z nodemon:watch chokidar watching: /app/lib/Database.js
2022-08-15T19:12:48.709Z nodemon:watch chokidar watching: /app/lib/Dev.js
2022-08-15T19:12:48.710Z nodemon:watch chokidar watching: /app/lib/Domain.js
2022-08-15T19:12:48.710Z nodemon:watch chokidar watching: /app/lib/Image.js
2022-08-15T19:12:48.711Z nodemon:watch chokidar watching: /app/lib/ImageDomain.js
2022-08-15T19:12:48.711Z nodemon:watch chokidar watching: /app/lib/Log.js
2022-08-15T19:12:48.712Z nodemon:watch chokidar watching: /app/lib/Reply.js
2022-08-15T19:12:48.712Z nodemon:watch chokidar watching: /app/lib/RouterHandler.js
2022-08-15T19:12:48.712Z nodemon:watch chokidar watching: /app/web/index.html

After this comes the long list from cache.

If i am looking in the debug logs after i remove the cache directory i get this :

2022-08-15T19:15:32.535Z nodemon:run spawn sh -c npm start || touch _keepalive.js
2022-08-15T19:15:32.537Z nodemon:run start watch on: [ '*.*', re: /.*\..*/ ]
2022-08-15T19:15:32.570Z nodemon:watch chokidar watching: /app/app.js
2022-08-15T19:15:32.571Z nodemon:watch chokidar watching: /app/config-sample.json
2022-08-15T19:15:32.572Z nodemon:watch chokidar watching: /app/config.json
2022-08-15T19:15:32.573Z nodemon:watch chokidar watching: /app/favicon.ico
2022-08-15T19:15:32.574Z nodemon:watch chokidar watching: /app/nodemon.json
2022-08-15T19:15:32.575Z nodemon:watch chokidar watching: /app/package.json
2022-08-15T19:15:32.582Z nodemon:watch chokidar watching: /app/lib/Database.js
2022-08-15T19:15:32.582Z nodemon:watch chokidar watching: /app/lib/Dev.js
2022-08-15T19:15:32.583Z nodemon:watch chokidar watching: /app/lib/Domain.js
2022-08-15T19:15:32.584Z nodemon:watch chokidar watching: /app/lib/Image.js
2022-08-15T19:15:32.585Z nodemon:watch chokidar watching: /app/lib/ImageDomain.js
2022-08-15T19:15:32.586Z nodemon:watch chokidar watching: /app/lib/Log.js
2022-08-15T19:15:32.586Z nodemon:watch chokidar watching: /app/lib/Reply.js
2022-08-15T19:15:32.587Z nodemon:watch chokidar watching: /app/lib/RouterHandler.js
2022-08-15T19:15:32.587Z nodemon:watch chokidar watching: /app/web/index.html
2022-08-15T19:15:51.698Z nodemon:watch filterAndRestart on [ 'package-lock.json' ]
2022-08-15T19:15:51.699Z nodemon:match rules [
  '!**/cache/**/*',
  '!**/logs/**/*',
  '!**/node_modules/**/*',
  '!**/package-lock.json',
  '!**/**/node_modules/**',
  '!**/**/coverage/**',
  '!**/**/bower_components/**',
  '!**/**/.sass-cache/**',
  '!**/**/.nyc_output/**',
  '!**/**/.git/**',
  '**/*.*'
]
2022-08-15T19:15:51.702Z nodemon:match ignored /app/package-lock.json rule: !**/package-lock.json
2022-08-15T19:15:51.702Z nodemon:match good []
2022-08-15T19:15:51.702Z nodemon:watch matched? {"result":[],"ignored":1,"watched":0,"total":1}

So here i got some unclear things :

remy commented 2 years ago

okay, quick test tells me the config isn't being loaded right. if you run nodemon -i cache it skips the cache directory properly.

remy commented 2 years ago

Sod, the debug should be this too: DEBUG=nodemon:*,nodemon <cmd> it gives more detail

remy commented 2 years ago

Change the nodemon.json to this:

{
  "verbose": true,
  "ignore": ["package-lock.json", "node_modules/","logs/","cache/"],
  "delay": 3000
 }

Note there's no "glob" at the end.

cberescu commented 2 years ago

ok, will do now , this is the extra log :

2022-08-15T19:49:25.568Z nodemon bus new listener: reset (0)
2022-08-15T19:49:25.571Z nodemon bus new listener: reset (0)
2022-08-15T19:49:25.582Z nodemon bus new listener: quit (0)
2022-08-15T19:49:25.583Z nodemon bus new listener: quit (0)
2022-08-15T19:49:25.583Z nodemon bus new listener: restart (0)
2022-08-15T19:49:25.583Z nodemon bus new listener: restart (0)
2022-08-15T19:49:25.583Z nodemon bus new listener: boot (0)
2022-08-15T19:49:25.583Z nodemon bus new listener: boot (0)
2022-08-15T19:49:25.586Z nodemon bus new listener: reset (2)
2022-08-15T19:49:25.588Z nodemon bus emit: boot
2022-08-15T19:49:25.589Z nodemon bus emit: reset
2022-08-15T19:49:25.589Z nodemon resetting watchers
2022-08-15T19:49:25.589Z nodemon reset
2022-08-15T19:49:25.624Z nodemon config: dirs [ '/app' ]
2022-08-15T19:49:25.630Z nodemon bus new listener: error (0)
2022-08-15T19:49:25.630Z nodemon bus new listener: error (0)
2022-08-15T19:49:25.632Z nodemon bus new listener: restart (2)
2022-08-15T19:49:25.632Z nodemon bus new listener: quit (2)
2022-08-15T19:49:25.632Z nodemon bus new listener: crash  (0)
2022-08-15T19:49:25.632Z nodemon bus new listener: crash  (0)
2022-08-15T19:49:25.632Z nodemon bus new listener: exit (0)
2022-08-15T19:49:25.632Z nodemon bus new listener: exit (0)
2022-08-15T19:49:25.642Z nodemon:run spawn sh -c npm start || touch _keepalive.js
2022-08-15T19:49:25.644Z nodemon bus new listener: exit (2)
2022-08-15T19:49:25.644Z nodemon:run start watch on: [ '*.*', re: /.*\..*/ ]
2022-08-15T19:49:25.645Z nodemon start watch on: /app
2022-08-15T19:49:25.645Z nodemon ignored [
  '**/.git/**',
  '**/.nyc_output/**',
  '**/.sass-cache/**',
  '**/bower_components/**',
  '**/coverage/**',
  '**/node_modules/**',
  'package-lock.json',
  'node_modules/*',
  'logs/*',
  'cache/*',
  re: /.*.*\/\.git\/.*.*|.*.*\/\.nyc_output\/.*.*|.*.*\/\.sass\-cache\/.*.*|.*.*\/bower_components\/.*.*|.*.*\/coverage\/.*.*|.*.*\/node_modules\/.*.*|package\-lock\.json|node_modules\/.*|logs\/.*|cache\/.*/
]
2022-08-15T19:49:25.674Z nodemon:watch chokidar watching: /app/app.js
2022-08-15T19:49:25.675Z nodemon:watch chokidar watching: /app/config-sample.json
2022-08-15T19:49:25.676Z nodemon:watch chokidar watching: /app/config.json
2022-08-15T19:49:25.676Z nodemon:watch chokidar watching: /app/favicon.ico
2022-08-15T19:49:25.677Z nodemon:watch chokidar watching: /app/nodemon.json
2022-08-15T19:49:25.678Z nodemon:watch chokidar watching: /app/package.json
2022-08-15T19:49:25.865Z nodemon:watch chokidar watching: /app/lib/Database.js
2022-08-15T19:49:25.866Z nodemon:watch chokidar watching: /app/lib/Dev.js
2022-08-15T19:49:25.867Z nodemon:watch chokidar watching: /app/lib/Domain.js
2022-08-15T19:49:25.867Z nodemon:watch chokidar watching: /app/lib/Image.js
2022-08-15T19:49:25.868Z nodemon:watch chokidar watching: /app/lib/ImageDomain.js
2022-08-15T19:49:25.868Z nodemon:watch chokidar watching: /app/lib/Log.js
2022-08-15T19:49:25.869Z nodemon:watch chokidar watching: /app/lib/Reply.js
2022-08-15T19:49:25.869Z nodemon:watch chokidar watching: /app/lib/RouterHandler.js
2022-08-15T19:49:25.870Z nodemon:watch chokidar watching: /app/web/index.html
cberescu commented 2 years ago

yeap, removing the "glob" fixed it

cberescu commented 2 years ago

Just confirming, but this is not really completed right? It is a bug. Or the documentation is missleading?

remy commented 2 years ago

Docs need to be clearer (probably in the faq)

On Mon, 15 Aug 2022, 23:48 cberescu, @.***> wrote:

Just confirming, but this is not really completed right? It is a bug. Or the documentation is missleading?

— Reply to this email directly, view it on GitHub https://github.com/remy/nodemon/issues/2057#issuecomment-1215951101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADLBB4LZR4BYPLAV375TDVZLCMJANCNFSM56S5DRGA . You are receiving this because you modified the open/close state.Message ID: @.***>

remy commented 2 years ago

I also note that the "perfect" version is to use **/cache/** this means that it'll have a more exact match to the path.

remy commented 2 years ago

I've added this to the readme, and changed the examples to use the explicit **/path/** - as there were a couple of examples using the method you used which I can see is going to mess people up!

cberescu commented 2 years ago

Thanks for the quick reply and all the help.

github-actions[bot] commented 1 year ago

:tada: This issue has been resolved in version 2.0.20 :tada:

The release is available on:

Your semantic-release bot :package::rocket: