HTML5 Drag and Drop with Selenium Webdriver

[lukeis]

Originally reported on Google Code with ID 3604

Are there any work arounds to getting HTML5 Drag and Drop working with Selenium Webdriver
with Ruby? I am using Selenium-Webdriver 2.20.0 with Ruby 1.9.2

Here is a simple test to reproduce the issue:

require "selenium-webdriver"
require "test/unit"

class Html5DragAndDropTest < Test::Unit::TestCase

  def setup
    @driver = Selenium::WebDriver.for :firefox
    @driver.manage.timeouts.implicit_wait = 30
    @verification_errors = []
  end

  def teardown
    @driver.quit
    assert_equal [], @verification_errors
  end

  def test_html5_drag_and_drop
    @driver.get("http://html5demos.com/drag")
    target = @driver.find_element(:id, "one")
    source = @driver.find_element(:id, "bin")
    @driver.action.drag_and_drop(target, source).perform
    assert target.displayed? == false
  end
end

Reported by rcorreia@blurb.com on 2012-03-21 19:59:17

[lukeis]

Reported by barancev on 2012-03-27 06:12:04

• Labels added: Component-WebDriver

[lukeis]

Here is a temporary workaround that could help the community with testing in the meantime...

1) drag_and_drop_helper.js to your test/helpers directory

2) Create a new method in your test_helper.rb

  def drag_and_drop(source,target)

    js_filepath=File.dirname(__FILE__)+"/drag_and_drop_helper.js"
    js_file= File.new(js_filepath,"r")
    java_script=""

    while (line=js_file.gets)
      java_script+=line
    end

    js_file.close

    @driver.execute_script(java_script+"$('#{source}').simulateDragDrop({ dropTarget:
'#{target}'});")

    rescue Exception => e
      puts "ERROR :" + e.to_s

  end

Reported by rcorreia@blurb.com on 2012-04-16 21:21:42

---

• _Attachment: drag_and_drop_helper.js_

[lukeis]

Where actually  this test/helpers directory located?. While running the script, I am
getting error (ERROR) jQuery is not defined

Reported by checkitoutkarthik on 2012-05-03 03:43:48

[lukeis]

this requires the implementation in the atoms of the events:

dragstart, dragover, dragenter, dragleave, drop

with native events this *could* work as is... although my attempt failed miserably.

Reported by luke.semerau on 2012-06-02 04:45:56

• Status changed: Accepted
• Labels added: Type-Enhancement, Browser-Atoms
• Labels removed: Type-Defect

[lukeis]

I'm seeing a similar issue with drag and drop using html 5 events. Can a developer confirm
that selenium drag and drop does not support html 5 drag and drop events?

Reference: http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html

Reported by catphive@catphive.net on 2012-09-12 02:14:51

[lukeis]

(i am a selenium developer) and yes, I'll confirm that html5 drag/drop is not currently
supported/implemented.

Reported by luke.semerau on 2012-09-12 04:46:15

[lukeis]

The workaround we put together is working for us.  It's been a life saver for testing
our Ember.js app.

attached is the latest version of what we are using...

this is what we have in our test_helper:

  def drag_and_drop(source,target)

    js_filepath=File.dirname(__FILE__)+"/drag_and_drop_helper.js"
    js_file= File.new(js_filepath,"r")
    java_script=""

    while (line=js_file.gets)
      java_script+=line
    end

    js_file.close

    @driver.execute_script(java_script+"$('#{source}').simulateDragDrop({ dropTarget:
'#{target}'});")

    rescue Exception => e
      puts "ERROR :" + e.to_s

  end

Reported by rcorreia@blurb.com on 2012-09-12 15:35:59

---

• _Attachment: drag_and_drop_helper.js_

[lukeis]

Issue 4455 has been merged into this issue.

Reported by a.u.savchuk on 2013-07-19 23:55:10

[lukeis]

Issue 5953 has been merged into this issue.

Reported by a.u.savchuk on 2013-07-19 23:56:18

[lukeis]

When will it be available in `selenium-webdriver` core?

Reported by tuka.08 on 2013-07-20 06:09:41

[lukeis]

Is there any planning date for this feature?
HTML 5 drag&drop is still not working and it will be very useful tho have it.

Reported by m.machniak on 2013-08-06 12:01:05

[lukeis]

Issue 5707 has been merged into this issue.

Reported by a.u.savchuk on 2013-08-13 19:03:43

[lukeis]

Issue 6222 has been merged into this issue.

Reported by a.u.savchuk on 2013-09-07 17:25:55

[lukeis]

Issue 6315 has been merged into this issue.

Reported by barancev on 2013-09-25 11:57:42

[lukeis]

I agree with m.machn... I would also be interested in whether there are any plans to
fix this bug?

Reported by mherrmann.at on 2013-09-25 12:12:38

[lukeis]

There is a workaround - the mouse just needs to move twice - for example once you click
and hold, move to any position on screen, then move to final destination, then release
and it works.

Reported by byatt.chris on 2013-09-25 12:26:48

[lukeis]

Thanks for your reply Byatt. I can't seem to get this workaround to work (Selenium 2.35.0,
FF 23.0.1, 64 bit Win 7). Here's what I tried:

    element_location = element.location
    to_location = to.location
    dx = to_location['x'] - element_location['x']
    dy = to_location['y'] - element_location['y']
    ActionChains(driver).click_and_hold(element).perform()
    ActionChains(driver).move_by_offset(dx / 2, dy / 2).perform()
    ActionChains(driver).move_to_element(to).perform()
    ActionChains(driver).release(to).perform()

Do you happen to have a working example?

Many thanks,
Michael

Reported by mherrmann.at on 2013-09-25 13:56:33

[lukeis]

I used a mixture of selenium and java robot. My code also allows for an x offset if
you want to drag it apst something. Should be easily modifiable.

public void dragAndDropElement(WebElement dragFrom, WebElement dragTo, int xOffset)
throws Exception {
        //Setup robot
        Robot robot = new Robot();
        robot.setAutoDelay(50);

        //Fullscreen page so selenium coordinates work
        robot.keyPress(KeyEvent.VK_F11);
        Thread.sleep(2000);

        //Get size of elements
        Dimension fromSize = dragFrom.getSize();
        Dimension toSize = dragTo.getSize();

        //Get centre distance
        int xCentreFrom = fromSize.width / 2;
        int yCentreFrom = fromSize.height / 2;
        int xCentreTo = toSize.width / 2;
        int yCentreTo = toSize.height / 2;

        //Get x and y of WebElement to drag to
        Point toLocation = dragTo.getLocation();
        Point fromLocation = dragFrom.getLocation();

        //Make Mouse coordinate centre of element
        toLocation.x += xOffset + xCentreTo;
        toLocation.y += yCentreTo;
        fromLocation.x += xCentreFrom;
        fromLocation.y += yCentreFrom;

        //Move mouse to drag from location
        robot.mouseMove(fromLocation.x, fromLocation.y);

        //Click and drag
        robot.mousePress(InputEvent.BUTTON1_MASK);

        //Drag events require more than one movement to register
        //Just appearing at destination doesn't work so move halfway first
        robot.mouseMove(((toLocation.x - fromLocation.x) / 2) + fromLocation.x, ((toLocation.y
- fromLocation.y) / 2) + fromLocation.y);

        //Move to final position
        robot.mouseMove(toLocation.x, toLocation.y);

        //Drop
        robot.mouseRelease(InputEvent.BUTTON1_MASK);
    }

Reported by byatt.chris on 2013-09-25 13:59:46

[lukeis]

Reported by barancev on 2013-11-02 17:39:46

[lukeis]

hi all,

or the proposed solution based on the drag_and_drop_helper.js script, is it possible
to use something similar to this to simulate dragging a set of files from outside of
the browser into an element?

Thx!

L.

Reported by leonardo.ramirez on 2014-01-06 17:32:49

[lukeis]

Hi,

after a lot of hard work, I eventually figured this out. It's implemented in a tool
of mine that makes tasks like this one-liners: http://heliumhq.com/docs/api_documentation#helium.api.drag_file.
No java.awt.robot is required, just any WebDriver object.

The idea for the implementation is the following: You need to simulate the following
JavaScript events:
i.  A `dragenter` event on the JavaScript document object.
ii. A `dragover` event on the JavaScript document object.
iii.    A `drop` event on the target element

All of these events include as a parameter a JavaScript `File` object that contains
information about the file being attached. The problem is that because of security
reasons, the browser does not allow creating such objects via JavaScript.

To work around the security constraints, we perform a little trick: We use JavaScript
to create an <input type="file"> element on the page. Then, we use Selenium's `send_keys`
command to send the path of the file to be dragged to this element. This automatically
sets a JavaScript property on the file input element that contains a `File` object
representing our file. Since we can read this property via JavaScript, we have succeeded
in obtaining the `File` object that is required to properly simulate the three events
above.

Hope this helps anyone wanting to implement this for himself.

Cheers,
Michael
heliumhq.com

Reported by michael.herrmann@open-closure.com on 2014-01-07 07:55:25

[lukeis]

We had trouble with the original drag_and_drop_helper.js posted as a workaround for
this issue.  The workaround is 99% correct, but I needed to modify the workaround to
include the dropTarget in the options propagated via the 'coord' object in simulateDrag.

i.e. I need to change:
  coord = { clientX: x, clientY: y }

to:
  coord = { clientX: x, clientY: y , dropTarget: options.dropTarget || undefined }

Also, a note for those following the example usage, if the app under test does not
already alias the jQuery function to $, you will need to spell-out jQuery:

i.e., after injecting the drag and drop helper onto the page, we have a method that
accepts jQuery selectors to use the simulated drag and drop functions (Java):

    /**
     * Drag and drop via the JQuery-based drag and drop helper -- the helper
     * must have been injected onto the page prior to calling this method.
     *
     * @param dragSourceJQuerySelector a JQuery-style selector that identifies the
source element to drag;
     * <em>will be passed directly to jQuery(), perform all quoting yourself</em>
     * @param dropTargetJQuerySelector a JQuery-style selector that identifies the
target element to drop the source onto;
     * <em>will be passed directly to jQuery(), perform all quoting yourself</em>
     */
    protected void dragAndDropViaJQueryHelper(String dragSourceJQuerySelector, String
dropTargetJQuerySelector) {
        String javascript =
            "var dropTarget = jQuery(" + dropTargetJQuerySelector + ");" +
            "\n" +
            "jQuery("+ dragSourceJQuerySelector + ").simulate('drag', { dropTarget:
dropTarget });";

        getLogger().info("executing javascript:\n" + javascript);
        this.executeScript(javascript);
        getLogger().info("executed drag-n-drop action via javascript");
    }

rcorr...@blurb.com thanks for sharing your workaround!

Reported by skuenzli on 2014-02-04 17:58:38

---

• _Attachment: drag_and_drop_helper.propagate-dropTarget.js_

[lukeis]

Thanks for the drag_and_drop_helper.js! It worked perfectly for my needs. You can see
a full write-up along with my example code here: http://elementalselenium.com/tips/39-drag-and-drop

Cheers,
Dave H
@TourDeDave

Reported by dave@arrgyle.com on 2014-02-26 17:09:49

[lukeis]

Just wondering, do we have similar issue with panning/rotating (or drag & drop'ing of
same element) in an HTML5 canvas? Wondering how you would do that from the current
workaround JS.

Reported by mangaroo on 2014-02-28 02:25:46

[lukeis]

Hi Everyone,

I tried to implement the solution however getting below exception org.openqa.selenium.WebDriverException:
jQuery(...).simulate is not a function.

String fileName = "C:/drag_and_drop_helper.propagate-dropTarget.js";
String injectScript = "var script = document.createElement(\"script\");";
injectScript += "script.src = \"" + fileName + "\";";
injectScript += "script.setAttribute(\"type\",\"text/javascript\");";
injectScript += "document.body.appendChild(script);";
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript(injectScript);
String javascript = "var dropTarget = jQuery($('#column-a'));" + "\n" + "jQuery($('#column-b')).simulate('drag',
{ dropTarget: dropTarget });";
        js.executeScript(javascript);

I got stuck at the moment. Request you all to provide any advice/suggestion.

Regards,
Anuj

Reported by anujbajaj10 on 2014-03-26 10:27:22

[lukeis]

Had you include the jquery.js into your page?

Reported by zhangwei8607 on 2014-03-29 05:00:01

[lukeis]

Hi,

I am able to get the above issue resolved however in my application under test, the
id is not unique for some elements, In this case, how shall we pass the properties
of the elements to drag & drop? For the below HTML, I am getting the error "jQuery
is not defined". Is there any other way to pass the elements properties such as xpath?

<img id="drag1" src="img_logo.gif" draggable="true" ondragstart="drag(event)" width="336"
height="69">

Reported by anujbajaj10 on 2014-03-31 08:30:02

[lukeis]

Can you define the element as a (complex) CSS selector? JQuery is just a variation of
a CSS selector.

But yea, it would be nice to be able to pass in a WebElement reference or use XPath,
since CSS (and I guess JQuery?) is limiting in some aspects like no text contains matching,
match by index for matches that return multiple results (e.g. (//xpath)[n] ), ancestor
& following/preceding sibling matching, etc.

Reported by mangaroo on 2014-03-31 18:11:45

[lukeis]

zhangwei/anujbaja, you can use <script src="path/to/your/jquery.js"></script>

anujbaja, if you dont have a unique way to locate your element you can always try using
the 'elements' method which returns an array of element objects.  Then you can just
use [n] to reference your element.

target = @driver.find_elements(:css, 'non_unique_locator')[0]
source = @driver.find_elements(:css, 'non_unique_locator')[1]

I would highly recommend using :css locators over :xpath as a best practice.

see also:
#SFSE: CSS Locators vs XPATH with Santiago Suarez Ordoñez
http://www.youtube.com/watch?v=6vPu3TO6XZ4

Hope that helps!

Reported by rcorreia@blurb.com on 2014-03-31 18:15:13

[lukeis]

manga,

CSS locators are capable of doing the things you're describing.  

Here is a good saucelabs post that should help you with your css locators:

http://saucelabs.com/resources/selenium/css-selectors

enjoy! 

Reported by rcorreia@blurb.com on 2014-03-31 18:51:15

[lukeis]

That's not quite correct. You can't do the following in CSS (and JQuery?):

(//someXPathThatReturnsMultipleMatches)[5]
returns 5 match out of all the returned matches, with CSS, you need to do find_elements()
then get 5th. With XPath, you can just call find_element() with that type of XPath
pattern and get the 5th match directly in one call

//div[contains(text(),'some text in the div')]
the CSS equivalent "div:contains('some text in div')" is a psuedo selector, and not
part of standard CSS3. Until fully adopted, it only works under JS libraries like Sizzle
(and maybe JQuery?). It worked in Selenium RC by default since that used Sizzle, it
doesn't work out of the box on WebDriver.

//div/preceding-sibling::div
CSS only offers following-sibling, effectively

//div/../../a
Can't navigate up in CSS only down the tree

//div[span[a[@id='blah']]]
which means find the div that contains a span that contains a hyperlink with id 'blah'.
It returns the matched div, not the hyperlink. I also think you can't define this in
CSS, but not sure.

Reported by mangaroo on 2014-03-31 18:55:32

[lukeis]

Thanks everyone for their suggestions.

I am asking about a specific scenario for drag & drop in HTML5 using jQuery helper
file. I am using Java as a language. Below is the sample code that I am trying to automate
however getting "org.openqa.selenium.WebDriverException: unknown error: jQuery is not
defined" exception. 

DesiredCapabilities capabilities = DesiredCapabilities.chrome();
driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capabilities);
driver.manage().window().maximize();
driver.get("http://html5demos.com/drag");
BufferedReader reader = new BufferedReader(new FileReader("C:/DargDrop.js"));
String line = null;
StringBuilder stringBuilder = new StringBuilder();
String ls = System.getProperty("line.separator");

while((line = reader.readLine()) != null) 
{
   stringBuilder.append(line);
   stringBuilder.append(ls);
}

JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript(stringBuilder.toString() + "$('#one').simulateDragDrop({ dropTarget:
'#bin'});");

Reported by anujbajaj10 on 2014-04-01 04:41:33

---

• Attachment: DargDrop.js

[lukeis]

To the above, jQuery simply isn't on the page then. Include it on the page.

To the others, jQuery uses Sizzle as it's selector backing. This allows jQuery to use
CSS selectors plus a few more additions, like contains. This means that a CSS selector
will work in jQuery but not necessarily the same is true for the other way round.

Reported by arran.huxtable on 2014-04-01 06:48:34

[lukeis]

Hi Arran,

Can you please let me know how to include jQuery on the page? Right now, I get stuck
at this point.

Appreciate your help on this.

Regards,
Anuj

Reported by anujbajaj10 on 2014-04-01 07:04:47

[lukeis]

Hi Everyone,

I will appreciate if anyone can give any suggestion. Below is the code that I am trying
however getting "Exception in thread "main" org.openqa.selenium.WebDriverException:
$ is not defined" exception.

js.executeScript("var jq = document.createElement('script');jq.src = 'C:/DargDrop.js';document.getElementsByTagName('head')[0].appendChild(jq);");

Thread.sleep(300);
js.executeScript("$('#one').simulateDragDrop({ dropTarget: '#bin'});");

Regards,
Anuj

Reported by anujbajaj10 on 2014-04-03 11:24:03

[lukeis]

Issue 7523 has been merged into this issue.

Reported by barancev on 2014-06-23 20:08:53

[lukeis]

Hi Dev team,

Please let me know any plan to fix this issue , as it's almost a yr old now :)

Many users are complaining about this since really long time.

kindly prioritize for next release of web driver.

Thanks in advance.

Kind Regards,
Vikram

Reported by vikram.silk on 2014-06-26 08:03:49

[lukeis]

Hello,

Has anyone managed to get HTML5 and drag and drop work in Junit and Java? I have seen
worked arounds in Ruby. But nothing in Java. When I ported ruby to JUnit I get an error
saying JQuery is not present...There is a post above asking about how to add Jquery
to a page if missing. I guess I have some more googling to do.

Reported by arad@qaworks.com on 2014-07-17 08:52:50

[lukeis]

Hi,

if using a commercial add-on to Selenium is an option for you then you can use Helium
(http://heliumhq.com). It includes a drag(...) function that also works with HTML5
(and doesn't require jQuery). The example from the top of this issue can for instance
be implemented in Java/JUnit as follows:

    import org.junit.After;
    import org.junit.Before;
    import org.junit.Test;

    import static com.heliumhq.API.*;
    import static org.junit.Assert.assertTrue;

    public class Html5DragTest {
        @Before
        public void setUp() {
            startChrome("http://html5demos.com/drag");
        }
        @Test
        public void testHtml5Drag() {
            // $(...) is a jQuery-style selector. Here it's used 
            // to select the HTML element with id "bin":
            drag("one", to($("#bin")));
            assertTrue(
                Text("nom").exists() || Text("gulp").exists() ||
                Text("yum!").exists() || Text("burp!").exists()
            );
        }
        @After
        public void tearDown() {
            killBrowser();
        }
    }

There's a free 30-day trial of Helium available at http://heliumhq.com/download. It
comes as a small Zip file. The only set up that's required to use it is to add with
some .jars to your classpath (http://heliumhq.com/docs/getting_started).

Reported by mherrmann.at on 2014-07-17 09:59:09

[lukeis]

In my case following code is enough to perform Drag&Drop in selenium webdriver Java
(in Chrome):

public static void DragAndDrop(WebElement LeftTreeElement, WebElement DestinationElement){
        Actions moveAndDrop = new Actions(driver);
        moveAndDrop.clickAndHold(LeftTreeElement).perform();
        moveAndDrop.moveToElement(DestinationElement).perform();
        moveAndDrop.release(DestinationElement).perform();

    }
Be sure that you obtain element in stable way.
I am using By selector or jquery selectors.

Regards,
Łukasz

Reported by Koscielny.Lukasz on 2014-07-17 10:08:15

[lukeis]

On #44, that works for HTML5-based drag & drop? And across browsers or just Chrome?
That seems like just standard drag & drop code, which is not meant for HTML5-based
drag and drop, which one has to implement the workarounds described in this issue thread.

Reported by mangaroo on 2014-07-17 17:38:54

[lukeis]

For those getting issue with jQuery (in Java) or not, see comment 25. Try swap out "$"
in the javascript code to execute with "jQuery". So instead of $('selectorValue'),
you do jQuery('selectorValue').

Reported by mangaroo on 2014-07-17 17:53:49

[lukeis]

below update from my side , still couldn't make it work

These 2 are dynamic objects which I need to deal with 

WebElement dragElement = getDriver().findElement(By.xpath("//div[@class='task-list']/ol[1]/li["+OriginalItemPositionToBeMoved+"]/div/div/span"));

WebElement dropElement = getDriver().findElement(By.xpath("//div[@class='task-list']/ol[1]/li["+TargetItemPositionTobeMoved+"]/div/div/span"));

//jsContent has got dnd.js as per blog http://elementalselenium.com/tips/39-drag-and-drop

String java_script = jsContent+"var dragElement = arguments[0], dropElement = arguments[1];";       

java_script = java_script + "jQuery(dragElement).simulateDragDrop({ dropTarget: dropElement});";

( (JavascriptExecutor)getDriver() ).executeScript(java_script, dragElement, dropElement);

But in above case , it's simply clicking first element and making it disappear , but
it's not dragging it to 2nd element's position.

anyone could make it work with java ?

Thanks,
Vikram

Reported by vikram.silk on 2014-07-23 08:56:11

[lukeis]

For the folks having issues with the drag & drop workaround (Java or not), it would
be good for folks to find/use a reference standard test site/page that is public where
this can be tested. Such that so if it works for me, it should work for you. And if
it doesn't work for your internal organizations site/app, then that's something app/implementation
specific since it works on the reference benchmark site/page.

And by the way, comment 25 is a working case for Java, although the provided code snippet
doesn't appear to be standalone complete code someone can just borrow and execute successfully.
One could try to contact that poster for details or help.

Reported by mangaroo on 2014-07-23 19:19:35

[lukeis]

If you review the issue you can see that I am proving the solution using: "http://html5demos.com/drag"

This site is a public demonstration of HTML5 Drag and Drop

Reported by rcorreia@blurb.com on 2014-07-23 22:43:40

[lukeis]

Hi all,

This page helped me get this script working so it is only fare to give a little back.

Please find attached working copy of Java HTML5 drag and drop.

Please note this code does not work with normal HTML as some of the internal HTML5
are missing.

"DirectJUNITDragdropHTML5.java" has the Java code.

You will need to also download "drag_and_drop_helper2.js" and in your Java code point
to it.

Reported by arad@qaworks.com on 2014-07-24 08:33:54

---

• Attachment: DirectJUNITDragdropHTML5.java
• _Attachment: drag_and_drop_helper2.js_

[lukeis]

#46 your solution did not work for me and I still get an exception. I have posted a
working solution above.

Please note that drag_and_drop_helper2.js is a slight modification of the same script
posted in this thread.

Reported by arad@qaworks.com on 2014-07-24 08:35:27

[lukeis]

This might be obvious but please make sure that your page is HTML5. Otherwise use normal
selenium drag and drop or click and move functionality. The code posted above will
not work against an HTML page. You will need the extra HTML5 functionality.

Reported by arad@qaworks.com on 2014-07-24 08:37:27

[lukeis]

Thanks for the contribution!  I will review and update the existing gist.

Reported by rcorreia@blurb.com on 2014-07-24 16:57:30

[lukeis]

Hi all,

Please note if your website has JQuery enabled then comment out the following line:
 Object o1 = js.executeScript(load_jquery);
from drag_and_drop_js function. this call is only necessary if your webpage does not
have JQuery loaded.

Running the above code if your site already uses JQuery risks changing the functionality
of the page for instance right click menu options.

Reported by arad@qaworks.com on 2014-07-25 09:20:36