Programmatically trigger Yomichan popups with JavaScript

KamWithK commented 1 year ago

Hey Yomichan team,

I was wondering whether there is a way to manually trigger a Yomichan lookup/popup over a word? Couldn't find any GitHub issues or anything online about this

Thanks for any help :)

toasted-nutbread commented 1 year ago

As it stands currently, probably not. Synthetic events may or may not work, but since scanning is entirely customizable, there's no way to figure out what event settings would be required.

KamWithK commented 1 year ago

I see, makes sense

Would you be able to give any advice on how to form a synthetic event for it? Or would it not be worth the effort?

toasted-nutbread commented 1 year ago

someElement.dispatchEvent(new MouseEvent('mousemove', {
  screenX: SOME_VALUE, // probably not needed
  screenY: SOME_VALUE, // probably not needed
  clientX: SOME_VALUE,
  clientY: SOME_VALUE,
  ctrlKey: false,
  shiftKey: true,
  altKey: false,
  metaKey: false,
  button: 0,
  buttons: 0,
  relatedTarget: null,
}));

This is assuming that the user's scanning configuration is to use the shift key + mouse motion to scan, which is not guaranteed. Ultimately, this would probably be something related to a Yomichan API, which has been brought up in #1649 previously, but hasn't been implemented.

KamWithK commented 1 year ago

Hmm I tried to do that but I can't exactly get the popup to appear (shift is my popup key)

For a nice and easy example I used a word from your message just now to test:

const test = document.querySelector(".pl-en")
const bound = test.getBoundingClientRect()

test.dispatchEvent(new MouseEvent('mousemove', {
//   screenX: bound.x + bound.left,
//   screenY: bound.y + bound.top,
  clientX: bound.left,
  clientY: bound.top,
  ctrlKey: false,
  shiftKey: true,
  altKey: false,
  metaKey: false,
  button: 0,
  buttons: 0,
  relatedTarget: null,
}))

Anything else you think I could quickly try? Or maybe this isn't really possible rn?

toasted-nutbread commented 1 year ago

You can try running this script on this page:

(() => {
  const node = document.querySelector('.comment-body>h3');
  const bound = node.getBoundingClientRect()
  const event = new MouseEvent('mousemove', {
    clientX: bound.left + 1, // These coordinates account for padding and such
    clientY: bound.bottom - 10,
    ctrlKey: false,
    shiftKey: true,
    altKey: false,
    metaKey: false,
    button: 0,
    buttons: 0,
    relatedTarget: null,
  });
  node.dispatchEvent(event);
})();

It should detect this text below (if it's on-screen):

読む

As you might be able to tell, the coordinates are fairly picky about what it considers to be hovering over the text.

Also, this seems to only work in Firefox.

KamWithK commented 1 year ago

Hmm interesting bc I'm on Firefox but running your code does nothing for me (no popup appears on the screen) I wonder why it works for you and not me

How did you find the padding values to use there btw? Maybe they can change between computers and so need to be found within the code?

toasted-nutbread commented 1 year ago

It's just kind of a guess based on what I was seeing in the inspector. Should be fairly similar across devices at least, not sure why it would work for me but not for you. Seems to work the same if I don't use the console and embed the code in a <script> tag on the page as well.

Update: it also seems to work in Chrome if I add bubbles: true to the event init options.

demo

KamWithK commented 1 year ago

That's even more interesting because I just tried Chrome with bubbles set to true and that had no effect either I can with mouse mouse activate that bubble but I can't with the console using your code

toasted-nutbread commented 1 year ago

Full standalone HTML page example below. Again, this relies on the scanning key being "shift".

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <title>Scan Test</title>
</head>
<body>

<h3 id="text">読む</h3>

<button id="test">Test</button>

<script>
document.querySelector('#test').addEventListener('click', () => {
  const node = document.querySelector('#text');
  const bound = node.getBoundingClientRect()
  const event = new MouseEvent('mousemove', {
    clientX: bound.left + 1, // These coordinates account for padding and such
    clientY: (bound.top + bound.bottom) * 0.5,
    ctrlKey: false,
    shiftKey: true,
    altKey: false,
    metaKey: false,
    button: 0,
    buttons: 0,
    relatedTarget: null,
    bubbles: true,
  });
  node.dispatchEvent(event);
}, 1000);
</script>

</body></html>

Demo: (left - Firefox, right - Chrome) demo2

KamWithK commented 1 year ago

Just gave that a go but nothing seems to happen at all (even tho I can shift and hover over it and it'll pop up) I don't think it's something to do with the screen size because even if i resize it the amount of padding/margin shouldn't really change much and you've taken the actual bound top/bottom/left values for it anyways

FooSoft / yomichan

Programmatically trigger Yomichan popups with JavaScript #2267

読む