assertj / assertj-swing

Fluent assertions for Swing apps
Other
109 stars 52 forks source link

Fix AssertJ Swing tests #230

Open croesch opened 6 years ago

croesch commented 6 years ago

Currently the test suite is not green. This might be related to #4 #5 #193 and #30

This is not only frustrating to me but for users willing to contribute. It prevents development of the framework and is not a healthy state.

croesch commented 6 years ago

Hint (thanks to @lbalazscs)

Recently I debugged a predictability issue in assertj-swing, maybe it helps you with the tests. I had some strange behavior that appeared only in some Linux tests. I run Linux in virtual machines, and only one of them was doing these weird things. After a lot of debugging I found that the problem is that after a simulated key is pressed, the OS starts to generate multiple key events until the simulated key release arrives, and sometimes too many events are generated even if in the code the releaseKey comes right after pressKey. For example

FrameFixture pw;
...
pw.pressKey(VK_CONTROL).pressKey(VK_Z)
                    .releaseKey(VK_Z).releaseKey(VK_CONTROL);

might generate two undo (Ctrl-Z) commands instead of just one, which ruins the testing logic. As a workaround I added to my tests the possibility to put events directly into the event queue (or execute the relevant code in other ways), see the osLevelKeyEvents flag in

Keyboard.java

Perhaps this idea (optionally putting the events into the event queue instead of using the Robot) could be implemented in the library itself, and it might help to debug the test issues.

croesch commented 6 years ago

The list of (sometimes) failing tests

AbstractButtonDriver_click_Test.should_Click_Disabled_Button expected:<LEFT_BUTTON> but was:<null>
AbstractButtonDriver_select_Test.should_Select_Button expected:<[tru]e> but was:<[fals]e>
AbstractButtonDriver_unselect_Test.should_Unselect_Button expected:<[fals]e> but was:<[tru]e>
AbstractContainerFixture_dialog_Test.should_Find_Visible_Dialog_By_Name_That_Needs_A_Couple_Of_Seconds_To_Appear:67 » WaitTimedOut
AbstractContainerFixture_fileChooser_Test.should_Find_Visible_JFileChooser_By_Matcher:115 » WaitTimedOut
AbstractContainerFixture_fileChooser_Test.should_Find_Visible_JFileChooser_By_Type:85 » WaitTimedOut
AbstractContainerFixture_optionPane_Test.should_Find_Visible_JOptionPane_By_Type_With_Timeout:56 » WaitTimedOut
ActivateWindowTask_activeWindow_Test.should_Activate_Window:65 » WaitTimedOut ...
Bug159_moveParentToFrontWhenClickingMenuItem_Test.should_Select_Menu_Item_From_Popup_Menu:78 » ActionFailed
Bug159_moveParentToFrontWhenClickingMenuItem_Test.should_Select_Menu_Item_From_Popup_Menu:79 » ActionFailed
Bug195_findDialogMultipleTimes_Test.should_Find_Dialog_Multiple_Times:54 » WaitTimedOut
Bug219_editTableCellWithEditorHavingCustomDocument_Test.should_Edit_Cell_Without_Activating_Error_Message:59 » ActionFailed
Bug232_wrongColumnIndexInJTable_Test.should_Enter_Value_After_Removing_Column:45 » ActionFailed
Bug289_robotShouldFocusWindowBeforeClosing_Test.should_Give_Focus_To_Window_Before_Closing_It expected:<[tru]e> but was:<[fals]e>
Bug293_errorWhenSelectingPathIfJTreeRootIsInvisible_Test.should_Select_Path_If_Root_Is_Visible:53 NullPointer
Bug76_JOptionPaneLookup_Test.should_Find_JOptionPane_Again:61 » WaitTimedOut T...
Bug80_JDialogLookup_Test.should_Find_Nested_Dialog_Two_Times:51->showAndHideNestedDialog:57 » WaitTimedOut
ComponentDriver_clickComponentAtPoint_Test.should_Click_Disabled_Component expected:<LEFT_BUTTON> but was:<null>
ComponentDriver_clickComponentWithMouseButton_Test.should_Click_Disabled_Component expected:<LEFT_BUTTON> but was:<null>
ComponentDriver_clickComponentWithMouseClickInfo_Test.should_Click_Disabled_Component expected:<LEFT_BUTTON> but was:<null>
ComponentDriver_doubleClick_Test.should_Double_Click_Disabled_Component expected:<LEFT_BUTTON> but was:<null>
ComponentDriver_invokePopup_Test.should_Show_JPopupMenu:30 » ComponentLookup U...
ComponentShownWaiter_waitTillShownWithCustomTimeout_Test.should_Wait_Till_Component_Is_Shown
FEST122_addSupportForEditableJComboBoxAsCellEditor_Test.should_Enter_Value_In_JTable_Cell_With_Editable_JComboBox_As_Editor [javax.swing.JTable[name='data', rowCount=6, columnCount=3, enabled=true, FEST130_wrongModifierBehaviorInJTableFixture_Test.should_Use_Shift_Modifier expected:<[1]> but was:<[0]>
FEST303_JFileChooserNotFoundOnMacOS_Test.should_Find_JFileChooser:48 » WaitTimedOut
FEST331_ShowPopupForNonFocusableComponents_Test.should_Show_JPopupMenu_For_NonFocusable_Component:46 » ComponentLookup
FEST65_JListFixtureNotSelectingItemsWithLongText_Test.shouldSelectItemWithSuperLongText
FrameDriver_maximize_Test.should_Maximize_Frame:29 » ActionFailed Platform doe...
FrameDriver_normalize_Test.should_Normalize_Frame expected:<[6]> but was:<[0]>
GitHub_190_FrameFixture_Test.test
Images_assertHasSize_Test.should_Fail_If_Actual_Is_Null
JOptionPaneFinder_findOptionPane_byType_Test.should_Find_JOptionPane:32 » WaitTimedOut
JOptionPaneFinder_findOptionPane_byType_Test.should_Find_JOptionPane_Before_Given_Timeout_Expires:40 » WaitTimedOut
JOptionPaneFinder_findOptionPane_withMatcher_Test.should_Find_JOptionPane:43 » WaitTimedOut
JTableDriver_selectCell_withManyCells_Test.shouldSelectCell[1] expected:<[tru]e> but was:<[fals]e>
JTableHeaderDriver_clickColumnByNameUsingButtonAndTimes_Test.should_Click_Column[1] expected:<LEFT_BUTTON> but was:<null>
JTableHeaderDriver_showPopupMenuByName_Test.should_Show_Popup_Menu_If_Column_Name_Matches_Pattern:49 » ComponentLookup
JToolBarDriver_floatTo_Test.should_Float_JToolBar
JToolBarDriver_unfloatWithConstraint_Test.should_Unfloat_JToolbar:55 » ActionFailed
JToolBarDriver_unfloatWithConstraint_Test.should_Unfloat_JToolbar:56 » ActionFailed
JTreeDriver_clickRow_Test.should_Click_Cell expected:<LEFT_BUTTON> but was:<null>
JTreeDriver_doubleClickPath_Test.should_Double_Click_Cell expected:<LEFT_BUTTON> but was:<null>
JTreeDriver_doubleClickRow_Test.should_Double_Click_Cell expected:<LEFT_BUTTON> but was:<null>
WindowAncestorFinder_windowAncestorOf_Test.should_Return_Invoker_As_Ancestor_Of_MenuElement:76 » ComponentLookup
WindowDriver_moveToBack_Test.should_Move_Window_To_Back:49 » TestTimedOut test...
WindowDriver_moveToFront_Test.should_Move_Window_To_Front expected:<[tru]e> but was:<[fals]e>
WindowFinder_findDialog_byName_Test.should_Find_Dialog:32 » WaitTimedOut Timed...
WindowFinder_findDialog_byType_Test.should_Find_Dialog:32 » WaitTimedOut Timed...
WindowFinder_findDialog_withMatcher_Test.should_Find_Dialog:44 » WaitTimedOut ...
WindowFinder_findFrame_byName_Test.should_Find_Frame:32 » WaitTimedOut Timed o...
WindowFinder_findFrame_byType_Test.should_Find_Frame:32 » WaitTimedOut Timed o...
WindowFinder_findFrame_withMatcher_Test.should_Find_Frame:44 » WaitTimedOut Ti...
WindowMoveToFrontTask_toFront_Test.should_Move_Window_To_Front expected:<[tru]e> but was:<[fals]e>