search

brackets-archive / bracketsIssues

Archive of issues in brackets.
0 stars 0 forks source link

[CLOSED] [integration tests] ProjectManager tests failing #5881

Open core-ai-bot opened 3 years ago

core-ai-bot commented 3 years ago

Issue by redmunds Friday Jan 10, 2014 at 23:52 GMT Originally opened as https://github.com/adobe/brackets/issues/6472

I'm seeing multiple tests fail, but the look very similar, so I'll only paste results of the first one. Test results are below. Both Mac and Win. UPDATE: latest issue only seems to happen on redmunds Windows 7 system.

Also, there are timeouts of 60000 which seem excessive. Is this really necessary?

UPDATE: Timeouts were reduced to 5000.

should create a new file with a given name

TypeError: Cannot read property 'name' of undefined
    at file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/project/ProjectManager.js:488:28
    at e (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/lodash.js:36:213)
    at Object.<anonymous> (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/project/ProjectManager.js:507:18)
    at Object.e (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/lodash.js:36:213)
    at x.isFunction.i (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/jquery-2.0.1.min.js:4:5488)
    at Array.sort (native)
    at Object.$.jstree.plugin._fn.sort (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:2154:48)
    at Object.$.jstree._fn.(anonymous function) [as sort] (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:234:19)
    at Object.<anonymous> (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:2142:12)
    at HTMLDivElement.x.isFunction.i (file://localhost/Users/randyedmunds/github/brackets-shell/xcodebuild/Release/Brackets.app/Contents/dev/src/thirdparty/jquery-2.0.1.min.js:4:5488)
timeout: timed out after 60000 msec waiting for ProjectManager.createNewItem() timeout
core-ai-bot commented 3 years ago

Comment by jasonsanjose Saturday Jan 11, 2014 at 00:00 GMT

Can you see if this is fixed with this PR https://github.com/adobe/brackets/pull/6468? There was a bug that was introduced recently when attempting to sort new untitled nodes in the tree.

core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 00:18 GMT

That fixes Mac, but not Windows.

These 60000 timeouts are killing me!

core-ai-bot commented 3 years ago

Comment by peterflynn Saturday Jan 11, 2014 at 00:28 GMT

@redmunds They should only change how long the test takes to run if the test is going to fail. In the normal case, the test should proceed as soon as ready regardless if the timeout is 1 sec or 10 years...

core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 00:38 GMT

@peterflynn Yes, but they are failing now and it's an unnecessarily long wait.

Any timeout should be directly related to the expected maximum length of the command that we're waiting for. I would say between 20 - 50% longer than the maximum expected length of time.

In these tests, we're creating new files or directories. What's the maximum expected time for that on the slowest disk supported by brackets? Maybe 1-2 seconds? Some of our users may be using slower network drives, but is anyone developing Brackets going to be using one?

core-ai-bot commented 3 years ago

Comment by peterflynn Saturday Jan 11, 2014 at 00:42 GMT

As others pointed out, the project tree refresh was slow enough in these tests that a 10+ sec timeout was indeed needed. Ian & Jason added a bunch of optimizations to the tree code since then, so once all that stuff lands in master we should see if the tests will run reliably with a shorter timeout. I bet 5-10 sec would be enough plenty at this point...

core-ai-bot commented 3 years ago

Comment by jasonsanjose Saturday Jan 11, 2014 at 01:48 GMT

@redmunds I pushed a few updates to jasonsanjose/file-tree-duplicates and the project manager tests pass for me on windows. Can you retry? Thanks!

core-ai-bot commented 3 years ago

Comment by iwehrman Saturday Jan 11, 2014 at 01:49 GMT

They're super fast now too :collision:

core-ai-bot commented 3 years ago

Comment by jasonsanjose Saturday Jan 11, 2014 at 01:49 GMT

@redmunds nevermind they're in master now

core-ai-bot commented 3 years ago

Comment by jasonsanjose Saturday Jan 11, 2014 at 01:53 GMT

FBNC@redmunds

core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 02:09 GMT

They're only fast when they pass :)

The ProjectManager tests pass on every time on Mac.

On Windows, they all passed the first time, but when I closed the Jasmine window, there was a Brackets test window that wasn't closed. So, I ran the tests again, and (after waiting what seemed like a really long time) there are 2 tests failing.

I verified that git status was clean before running tests again. Same results each time now.

should fail when a file name matches a directory that already exists

TypeError: Cannot call method 'lastIndexOf' of undefined
    at getBaseName (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/file/FileUtils.js:250:34)
    at getFileExtension (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/file/FileUtils.js:302:24)
    at Object.compareFilenames (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/file/FileUtils.js:373:22)
    at Object.<anonymous> (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/project/ProjectManager.js:497:26)
    at Object.e (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/thirdparty/lodash.js:36:213)
    at x.isFunction.i (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/thirdparty/jquery-2.0.1.min.js:4:5488)
    at Array.sort (native)
    at Object.$.jstree.plugin._fn.sort (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:2154:48)
    at Object.$.jstree._fn.(anonymous function) [as sort] (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:234:19)
    at Object.<anonymous> (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/src/thirdparty/jstree_pre1.0_fix_1/jquery.jstree.js:2142:12)

timeout: timed out after 60000 msec waiting for ProjectManager.createNewItem() timeout

should fail when file name contains special characters

Error: Expected 2 to be 1.
    at new jasmine.ExpectationResult (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:102:32)
    at null.toBe (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1194:29)
    at Object.clickDialogButton (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/spec/SpecRunnerUtils.js:457:29)
    at assertFile (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/spec/ProjectManager-test.js:175:37)
    at jasmine.Block.execute (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1024:15)
    at jasmine.Queue.next_ (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1842:31)
    at jasmine.Queue.start (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1795:8)
    at jasmine.Spec.execute (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:2122:14)
    at jasmine.Queue.next_ (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1842:31)
    at onComplete (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1838:18)

Error: Expected 2 to be 1.
    at new jasmine.ExpectationResult (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:102:32)
    at null.toBe (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1194:29)
    at Object.clickDialogButton (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/spec/SpecRunnerUtils.js:461:38)
    at assertFile (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/spec/ProjectManager-test.js:175:37)
    at jasmine.Block.execute (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1024:15)
    at jasmine.Queue.next_ (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1842:31)
    at jasmine.Queue.start (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1795:8)
    at jasmine.Spec.execute (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:2122:14)
    at jasmine.Queue.next_ (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1842:31)
    at onComplete (file:///C:/Users/redmunds/github/brackets-shell/Release/dev/test/thirdparty/jasmine-core/jasmine.js:1838:18)
core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 02:11 GMT

Removed FBNC tag. Added "Win only" tag. Back to@jasonsanjose cc:@iwehrman

core-ai-bot commented 3 years ago

Comment by jasonsanjose Saturday Jan 11, 2014 at 02:15 GMT

It looks like your running a copy of the brackets repo from the staging folder in brackets-shell. Are you sure those files are up to date? Those line numbers (on master) from the stack traces don't look valid.

core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 02:55 GMT

I forgot to paste these messages from console:

[node-error 5:59:21 PM] Error watching file C:/Users/redmunds/github/brackets/test/temp/: watch EPERM NodeDebugUtils.js:114
handleLogEvent NodeDebugUtils.js:114
Unable to stat changed path:  C:/Users/redmunds/github/brackets-shell/Release/dev/test/temp/toDelete1/file.js NotFound AppshellFileSystem.js:74
[node-error 5:59:34 PM] Error watching file C:/Users/redmunds/github/brackets-shell/Release/dev/test/temp/directory/: watch EPERM NodeDebugUtils.js:114
handleLogEvent NodeDebugUtils.js:114
Unable to stat changed path:  C:/Users/redmunds/github/brackets-shell/Release/dev/test/temp/directory NotFound AppshellFileSystem.js:74
core-ai-bot commented 3 years ago

Comment by redmunds Saturday Jan 11, 2014 at 17:40 GMT

UPDATE: I rebooted my Windows machine, and ProjectManager tests passed first time, but failed (same as above) on subsequent runs. I verified git status was clean before each run.

Interesting that if I shutdown/restart Brackets, verify git status is clean, even delete cef_data, tests fail on first run. If problem was related to tests not cleaning up something, then I would expect it to pass first time after restart.

core-ai-bot commented 3 years ago

Comment by redmunds Sunday Jan 12, 2014 at 19:40 GMT

@jasonsanjose@iwehrman The error is in this code:

        var a1 = $(a).data("compareString"),
            b1 = $(b).data("compareString");

        return FileUtils.compareFilenames(a1, b1, false);

Either a or b does not (yet) have "compareString" set, so it seems to be a race condition.

core-ai-bot commented 3 years ago

Comment by jasonsanjose Tuesday Jan 14, 2014 at 21:11 GMT

@redmunds can you still reproduce these failures on master?

core-ai-bot commented 3 years ago

Comment by redmunds Tuesday Jan 14, 2014 at 22:34 GMT

@jasonsanjose Yes, I pulled the latest master and I'm still see these failures on Win7. Tests pass on Mac 10.8.

core-ai-bot commented 3 years ago

Comment by jasonsanjose Wednesday Jan 15, 2014 at 00:57 GMT

Hmm, I can't reproduce those failures on windows. Maybe clean out prefs or temp dir?

core-ai-bot commented 3 years ago

Comment by redmunds Wednesday Jan 15, 2014 at 02:07 GMT

Got latest master, deleted brackets.json & cef_data, verified 'git status' was clean and tests passed!

I switched to PreferencesModel branch, and they failed again. cc@dangoor

I switched back to master, and they failed again. Deleted cef_data and brackets.json. and test still fail.

Can we merge #6479 (with timeouts of 5000) so this doesn't take so long?

After that, can you try it in the PreferencesModel branch?

core-ai-bot commented 3 years ago

Comment by dangoor Wednesday Jan 15, 2014 at 02:57 GMT

@redmunds do the ProjectManager tests fail for you when you run only those tests, or do you have to run the whole suite?

For me, the ProjectManager tests pass when I run that suite individually on both master and PreferencesModel.

core-ai-bot commented 3 years ago

Comment by redmunds Wednesday Jan 15, 2014 at 03:58 GMT

They fail when I run only the ProjectManager tests.

core-ai-bot commented 3 years ago

Comment by redmunds Wednesday Jan 15, 2014 at 16:41 GMT

I seem to be the only one having this problem, so I changed Priority to Medium and removed Sprint 36 Milestone.

core-ai-bot commented 3 years ago

Comment by redmunds Tuesday Jan 21, 2014 at 19:06 GMT

@jasonsanjose I'm not sure exactly what fixed it, but I think it was a change@dangoor made today for PreferencesModel. Closing.