search

electron / asar

Simple extensive tar-like archive format with indexing
MIT License
2.55k stars 248 forks source link

Order of files in archive varies each time #6

Closed bwin closed 9 years ago

bwin commented 9 years ago

This makes testing impossible.

bwin commented 9 years ago

:smile_cat: wrong title at first.

bwin commented 9 years ago

This is because of walkdir. Filed an issue there https://github.com/soldair/node-walkdir/issues/17

You can easily reproduce the issue with a directory with 2 files in it. Generate an archive from it a few times and compare the output.

bwin commented 9 years ago

Some tests for asar would be nice. This blocks adding tests.

@zcbenz How about switching from walkdir to glob (or something else)? I could send you a PR, but you have to be ok with this.

I would really like to add some simple tests.

zcbenz commented 9 years ago

I remember some troubles when using glob on Windows, but if it works fine now I think it is OK to switch to it.

bwin commented 9 years ago

Looking at atom-shell, it seems you write tests BDD-style (describe, it, ...) with assert. What test framework do you prefer? Would mocha be ok?

zcbenz commented 9 years ago

mocha is fine, I don't really have a preference.

YurySolovyov commented 9 years ago

Hey @bwin are you still going to submit a PR about replacing walkdir with glob ? It seems to be reliable and better maintaied. I also solved some data races in my build process, with https://github.com/bwin/asar/commit/308d5cae8a4698543c97cd8fbf0da77d39172b19

bwin commented 9 years ago

Hi Yuri, I settled for sorting the walkdir results (it's faster than glob). That's already merged.

bwin commented 9 years ago

I'm also still working on a complete rewrite of this repo (in tandem with the asar-part in atom-shell)...

Edit: BTW it already took me way too long, I know that. Sometimes soo much stuff happens...

bwin commented 9 years ago

I also solved some data races in my build process, with bwin@308d5ca

Please tell me more.

YurySolovyov commented 9 years ago

I just had an issue with asar JS API not working correctly: sometimes it not included some files or folders in archive, so I tried to replace walkdir with glob and it seems to work better, but I need to more time with it to be sure.

I'm also still working on a complete rewrite of this repo (in tandem with the asar-part in atom-shell)...

Are you goind to submit a PR or something? or just fork?

Please tell me more.

I'm using this script to build my app. Sometimes it builds just fine, and sometimes some files are missing. I'm still not sure what causes it but again, it seems to work better with glob.

bwin commented 9 years ago

Sorry for the delay, went out to eat something.

Are you going to submit a PR or something? or just fork?

PR (so it's zcbenz who decides if it makes it in, and if not then it'll be fork)

Sometimes it builds just fine, and sometimes some files are missing.

There are two things I can think of (right now) that can go wrong.

Ok, I looked at bwin@308d5ca and saw this:

disk.writeFilesystem(dest, filesystem, files);
//...

// vs.

disk.writeFilesystem(dest, filesystem, files, function() { //...

The callback gets called prematurely (without waiting for disk.writeFilesystem to finish). I'll fix it. Hmm, it's already fixed in master. Edit: It was fixed in bwin@e5cf869, so if you were using something older, then that's most certainly the cause. If you are on a newer asar and still experience these problems, I can look into it (please open a new issue).

Do you use the latest version of asar? Probably.

YurySolovyov commented 9 years ago

@bwin

You reach some kind of path-depth limit. (But that would probably fail every time.)

no, folder structure is pretty flat, 5 nesting levels max.

Or the asar API calls your callback too early.

This might be the case, because I don't see any pattern in what files got skipped.

Do you use the latest version of asar? Probably.

0.2.1