SeleniumHQ / selenium-ide

Open Source record and playback test automation for the web.
https://selenium.dev/selenium-ide/
Apache License 2.0
2.73k stars 741 forks source link

Mouse coordinates for canvas interaction #1128

Open EmileSonneveld opened 3 years ago

EmileSonneveld commented 3 years ago

🚀 Feature Proposal

When clicking on a canvas element, the location of the mouse is not registered correctly. For example, here clicking inside or outside the rectangle of a canvas gives different behaviour: https://emilesonneveld.be/dropbox_proxy/dev_freetime/interactive_canvas.html

However, the generated code will always click on the same location of the canvas :-(

<html>
  <body>
    <canvas style="background-color: gray;"></canvas>
    <p>Clicked inside or outside of rectangle?<br/></p>
    <script type="text/javascript">
      var logs = document.querySelector("p")
      var canvas = document.querySelector("canvas")
      var ctx = canvas.getContext("2d")

      var rect = {
        x: 20,
        y: 20,
        width: 150,
        height: 100,
      }
      ctx.beginPath()
      ctx.rect(rect.x, rect.y, rect.width, rect.height)
      ctx.fill()

      canvas.addEventListener("click", function(e){
        console.log(e)
        if(e.offsetX > rect.x
          && e.offsetY > rect.y
          && e.offsetX < rect.x + rect.width
          && e.offsetY < rect.y + rect.height){
          ctx.fillStyle = "green"
          logs.innerHTML += "clicked inside rectangle<br/>"
        }else{
          ctx.fillStyle = "red"
          logs.innerHTML += "clicked outside rectangle<br/>"
        }
        ctx.beginPath()
        ctx.rect(rect.x, rect.y, rect.width, rect.height)
        ctx.fill()
      })
    </script>
  </body>
</html>

Motivation

Learning to write selenium scripts is time consuming. This way, anything can be automated. :D I won't mind if recording mouse positions makes the test weight 500Mb. Development time is way more expensive than hard disk space. If making automated tests is easy enough, it could get priority in the company :)

Example

We have a website with many different canvases where the user has to interact with. If we could easily automate most of the interactions, it would save us a whole lot of hassle.

toddtarsi commented 1 year ago

I will keep an eye on this in v4. I believe an interesting technique a friend of mine uses on SeleniumBase is to generate alternate commands by using hotkeys