huginn / huginn

Create agents that monitor and act on your behalf. Your agents are standing by!
MIT License
43.11k stars 3.75k forks source link

Website Agent - Possible to only look at sub-section of website? Or something else? #2663

Closed dohlin closed 4 years ago

dohlin commented 4 years ago

Hey guys...I'm sorry this is yet another website agent question. Tried searching but couldn't find anything exactly like what I'm trying to do so either my Google-fu is failing me or something. Anyways, hoping this is a simple fix and someone here can help!

I'm trying to watch the website https://www.connectwise.com/software/control/download (specifically the "Pre-Release" and "Stable Release" sections, but the whole page if necessary) but the following Huginn code alerts me every time it runs. I've tried several different "extract" options too such as css: html and value: . but not luck.

{
  "expected_update_period_in_days": "7",
  "url": "https://www.connectwise.com/software/control/download",
  "type": "html",
  "mode": "on_change",
  "extract": {
    "text_content": {
      "css": "body",
      "value": "normalize-space(.)"
    }
  }
}

If I put 2 subsequent events through a diff checker online I can see that they're almost identical, except for at the very end there is a line that reads "2020-01-08 12:00:08 - IP-0A0A1631 AngularInit.bootstrap();" where the bolded time is the time that Huginn agent last ran at...which obviously creates a different result each time.

What can I do here or what am I missing? Is it possible to exclude this section, and/or to monitor only the 2 above mentioned "sub-sections" of this site? If so how would that options code within Huginn look? Thank you!

dohlin commented 4 years ago

Nevermind...think I got it figured out. I am not a developer nor an HTML expert so I wasn't sure how to find and use the "xpath" options. With Chrome developer tools though I was able to right click > inspect the page and find the section I wanted to watch, and then copy the corresponding xpath code. For anyone who happens across this down the road and wants another example of what worked for me, the following code in Huginn is what I ended up with. Thanks!

{
  "expected_update_period_in_days": "7",
  "url": "https://www.connectwise.com/software/control/download",
  "type": "html",
  "mode": "on_change",
  "extract": {
    "Pre-Release": {
      "xpath": "/html/body/main/section[6]",
      "value": "normalize-space(.)"
    },
    "Stable Release": {
      "xpath": "/html/body/main/section[8]",
      "value": "normalize-space(.)"
    }
  }
}