sometimes cpx fails to copy files and directories

mysticatea / cpx

A cli tool to watch and copy file globs.

MIT License

524 stars 36 forks source link

sometimes cpx fails to copy files and directories #29

Open akosyakov opened 7 years ago

akosyakov commented 7 years ago

sometimes chokidar should be tuned to a concrete OS to work reliable, be able to pass chokidar options will be nice

akosyakov commented 7 years ago

Also migrating to the latest chokidar also would be nice

mysticatea commented 7 years ago

Thank you for this issue.

Could you make a concrete proposal? I can work in this weekend.

Also migrating to the latest chokidar also would be nice

^1.6.0 looks to install the latest version: https://david-dm.org/mysticatea/cpx

akosyakov commented 7 years ago

Sometimes not all files are copied for some reasons, we want to try to pass awaitWriteFinish.

akosyakov commented 7 years ago

@mysticatea Maybe you can actually fix real issues:

cpx sometimes copy files which are not completely written yet, only partial content
if a glob src pattern can be matched to a folder, like src/**/*, then sometimes cpx completely misses copying subfolders

You can see more discussions on this PR: https://github.com/theia-ide/theia/pull/156

The issue seems to be related to when and how fast events are produced by chokidar.

I see that vscode buffers events from chokidar and only when 50 milliseconds passed and there were no new events from chokidar it propagates buffered to the client code: https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/services/files/node/watcher/unix/chokidarWatcherService.ts#L25. It could be that it is done to overcome issues like stated above.

Also awaitWriteFinish was not helpful to overcome it.

Right now it is a showstopper issue for us to continue using cpx.

mysticatea commented 7 years ago

Thank you for the information. I'm sorry that I failed to get time in the last weekend because of my job was busy. I'll work within some days, I have 4 days' holiday since 6/1.

akosyakov commented 7 years ago

I've ended up using gulp 4 instead for watching and copying. It uses https://github.com/gulpjs/glob-watcher for watching that behaves quite deterministic. And for copying one can use newer plugin to copy incrementally that boosts performance.

bobbylight commented 6 years ago

@akosyakov, is this the reason I sometimes see files not copied, but a 0-byte "copy" created in the target directory instead?

mdmoreau commented 6 years ago

@bobbylight @mysticatea I'm noticing the same behavior. I raised a similar issue about postcss-cli (https://github.com/postcss/postcss-cli/issues/215) which was fixed by adjusting the chokidar options as mentioned. Babel has a similar fix in place at https://github.com/babel/babel/blob/ab62a9439918edf97eeea730a444a622542696ff/packages/babel-cli/src/babel/dir.js#L152-L155, so it seems to be a common workaround.

SCLeoX commented 4 years ago

Is this ever going to be fixed? For me it fails about 40% of the time.

Is there any replacement for this module if it is not going to be fixed?