eficode / robotframework-imagehorizonlibrary

Cross-platform Robot Framework library for GUI automation based on image recognition
MIT License
79 stars 26 forks source link

Image Keywords not working on Mac #52

Open dschiller opened 3 years ago

dschiller commented 3 years ago

Following Keywords fail:

Locate
Wait for
Click Image

Error is:

ImageNotFoundException: Reference image "myimage.png" was not found on screen

If I use an Image inside VSCode, where I also execute the Robot Tests then ImageHorizonLibrary find the Image. If I use any Image which is not part of VSCode ( eg. Finder, any Apps, etc. ) then ImageHorizonLibrary will NOT find the Image provided and result in ImageNotFoundException even if the App which I used to take a Screenshot from has the Focus.

I also saw that there will be created a Screenshot TestLogicAUQT-screenshot-1.png. How is that working ?

image

This Image shows an empty Desktop with Logic Pro active not showing any Dialogs.

image

If I change to Finder while Test is running I get the same empty Screenshot showing the Desktop only Finder is focused then. It seems the ImageHorizonLibrary is not able to see Dialogs / Windows of an App.

Is there a way to also use Images outside of VSCode ?

Cheers Dirk

Code Example

TestLogicAUQT.robot

*** Settings ***
Library  Libraries/LibDaw.py
Library  ImageHorizonLibrary  reference_folder=${CURDIR}${/}Data${/}UIAutomationImages

*** Test Cases ***
Test Logic Pro AU QT
  Open DAW  Logic Pro X
  Locate  LPX_EmptyProject  # <-- Doesn't work; Screenshot from Logic
  Wait for  Tutorials  # <-- Doesn't work; Screenshot from Logic
  Wait for  Facts  # <-- Works as it is a Screenshot from VScode
  Wait for  Facts2  # <-- Doesn't work; Screenshot from Finder

Versions

MacOS Catalina 10.15.7 (19H524) on Intel
Python 3.8.7
MouseInfo==0.1.3
Pillow==8.1.2
PyAutoGUI==0.9.52
PyGetWindow==0.0.9
PyMsgBox==1.0.9
pyobjc==7.1
pyobjc-core==7.1
pyobjc-framework-Accounts==7.1
pyobjc-framework-AddressBook==7.1
pyobjc-framework-AdSupport==7.1
pyobjc-framework-AppleScriptKit==7.1
pyobjc-framework-AppleScriptObjC==7.1
pyobjc-framework-ApplicationServices==7.1
pyobjc-framework-AuthenticationServices==7.1
pyobjc-framework-AutomaticAssessmentConfiguration==7.1
pyobjc-framework-Automator==7.1
pyobjc-framework-AVFoundation==7.1
pyobjc-framework-AVKit==7.1
pyobjc-framework-BusinessChat==7.1
pyobjc-framework-CalendarStore==7.1
pyobjc-framework-CFNetwork==7.1
pyobjc-framework-CloudKit==7.1
pyobjc-framework-Cocoa==7.1
pyobjc-framework-Collaboration==7.1
pyobjc-framework-ColorSync==7.1
pyobjc-framework-Contacts==7.1
pyobjc-framework-ContactsUI==7.1
pyobjc-framework-CoreAudio==7.1
pyobjc-framework-CoreAudioKit==7.1
pyobjc-framework-CoreBluetooth==7.1
pyobjc-framework-CoreData==7.1
pyobjc-framework-CoreHaptics==7.1
pyobjc-framework-CoreLocation==7.1
pyobjc-framework-CoreMedia==7.1
pyobjc-framework-CoreMediaIO==7.1
pyobjc-framework-CoreMIDI==7.1
pyobjc-framework-CoreML==7.1
pyobjc-framework-CoreMotion==7.1
pyobjc-framework-CoreServices==7.1
pyobjc-framework-CoreSpotlight==7.1
pyobjc-framework-CoreText==7.1
pyobjc-framework-CoreWLAN==7.1
pyobjc-framework-CryptoTokenKit==7.1
pyobjc-framework-DeviceCheck==7.1
pyobjc-framework-DictionaryServices==7.1
pyobjc-framework-DiscRecording==7.1
pyobjc-framework-DiscRecordingUI==7.1
pyobjc-framework-DiskArbitration==7.1
pyobjc-framework-DVDPlayback==7.1
pyobjc-framework-EventKit==7.1
pyobjc-framework-ExceptionHandling==7.1
pyobjc-framework-ExecutionPolicy==7.1
pyobjc-framework-ExternalAccessory==7.1
pyobjc-framework-FileProvider==7.1
pyobjc-framework-FileProviderUI==7.1
pyobjc-framework-FinderSync==7.1
pyobjc-framework-FSEvents==7.1
pyobjc-framework-GameCenter==7.1
pyobjc-framework-GameController==7.1
pyobjc-framework-GameKit==7.1
pyobjc-framework-GameplayKit==7.1
pyobjc-framework-ImageCaptureCore==7.1
pyobjc-framework-IMServicePlugIn==7.1
pyobjc-framework-InputMethodKit==7.1
pyobjc-framework-InstallerPlugins==7.1
pyobjc-framework-InstantMessage==7.1
pyobjc-framework-Intents==7.1
pyobjc-framework-IOSurface==7.1
pyobjc-framework-iTunesLibrary==7.1
pyobjc-framework-LatentSemanticMapping==7.1
pyobjc-framework-LaunchServices==7.1
pyobjc-framework-libdispatch==7.1
pyobjc-framework-LinkPresentation==7.1
pyobjc-framework-LocalAuthentication==7.1
pyobjc-framework-MapKit==7.1
pyobjc-framework-MediaAccessibility==7.1
pyobjc-framework-MediaLibrary==7.1
pyobjc-framework-MediaPlayer==7.1
pyobjc-framework-MediaToolbox==7.1
pyobjc-framework-Metal==7.1
pyobjc-framework-MetalKit==7.1
pyobjc-framework-MetalPerformanceShaders==7.1
pyobjc-framework-ModelIO==7.1
pyobjc-framework-MultipeerConnectivity==7.1
pyobjc-framework-NaturalLanguage==7.1
pyobjc-framework-NetFS==7.1
pyobjc-framework-Network==7.1
pyobjc-framework-NetworkExtension==7.1
pyobjc-framework-NotificationCenter==7.1
pyobjc-framework-OpenDirectory==7.1
pyobjc-framework-OSAKit==7.1
pyobjc-framework-OSLog==7.1
pyobjc-framework-PencilKit==7.1
pyobjc-framework-Photos==7.1
pyobjc-framework-PhotosUI==7.1
pyobjc-framework-PreferencePanes==7.1
pyobjc-framework-PushKit==7.1
pyobjc-framework-Quartz==7.1
pyobjc-framework-QuickLookThumbnailing==7.1
pyobjc-framework-SafariServices==7.1
pyobjc-framework-SceneKit==7.1
pyobjc-framework-ScreenSaver==7.1
pyobjc-framework-ScriptingBridge==7.1
pyobjc-framework-SearchKit==7.1
pyobjc-framework-Security==7.1
pyobjc-framework-SecurityFoundation==7.1
pyobjc-framework-SecurityInterface==7.1
pyobjc-framework-ServiceManagement==7.1
pyobjc-framework-Social==7.1
pyobjc-framework-SoundAnalysis==7.1
pyobjc-framework-Speech==7.1
pyobjc-framework-SpriteKit==7.1
pyobjc-framework-StoreKit==7.1
pyobjc-framework-SyncServices==7.1
pyobjc-framework-SystemConfiguration==7.1
pyobjc-framework-SystemExtensions==7.1
pyobjc-framework-UserNotifications==7.1
pyobjc-framework-VideoSubscriberAccount==7.1
pyobjc-framework-VideoToolbox==7.1
pyobjc-framework-Vision==7.1
pyobjc-framework-WebKit==7.1
pyperclip==1.8.2
PyRect==0.1.4
PyScreeze==0.1.26
PyTweening==1.0.3
robotframework==3.1.1
robotframework-imagehorizonlibrary==1.0
rubicon-objc==0.4.0
dschiller commented 3 years ago

Tryed to use robotframework-SikuliLibrary instead but same Issue there.

To fix the Issue in Sikuli I need to change the Screenshot Image Size to 50 Percent.

I found out that the Screenshot Tool on Mac saves the images in it's double size. To fix that open the Screenshot taken with the Mac internal Tools in Preview and change it's size ( Tools / Adjust Size ) to 50 Percent and save. This way SikuliLibrary is able to find the Screenshots on the screen.

Unfortunetly this does NOT fix the ImageHorizonLibrary Issues.