rstudio / chromote

Chrome Remote Interface for R
https://rstudio.github.io/chromote/
156 stars 20 forks source link

Problem with chromote package #159

Open sale4cast opened 4 months ago

sale4cast commented 4 months ago

Dear community,

First of all, I would like to thanks everyone for your previous support. I want to extract "hotel prices" from google. From your previous suggestions, I can extract the first page prices perfectly. However, I want to extract also second page prices. Here I attach the code below:

library(shiny)
library(curl)
library(chromote)
library(pagedown)

ui <- fluidPage(
  textOutput("targetId"),
  tableOutput("result")
)

r <- Chromote$new()

server <- function(session, input, output) {
  driver <- r$new_session()
  output$targetId <- renderText(paste0("Target ID: ", driver$get_target_id()))
  p <- driver$Page$loadEventFired(wait_ = FALSE)
  driver$Page$navigate("https://google.com", wait_ = FALSE)

  p$then(function(value){
    googleSearchText <- "4 star hotel in barcelona"
    driver$Runtime$evaluate(paste0('document.querySelector("textarea").value = "', googleSearchText,'"'))

    {
      p2 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("input[aria-label=\'Google Search\']").click()', wait_ = FALSE)
    }

    p2
  })$then(function(value){
    driver$Runtime$evaluate('document.querySelector("div.R2w7Jd").click()')
    {
      p3 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("div.JWXKNd").click()', wait_ = FALSE)
    }
    p3
  })$then(function(value){
    priceElement <- driver$Runtime$evaluate(
      'var elements = document.querySelectorAll(".K1smNd > c-wiz[jsrenderer=\'hAbFdb\'] .PwV1Ac");
                 var elementPrices = [];
                 elements.forEach(function(element) {
                   elementPrices.push(element.innerText);
                 });
                 elementPrices.join("@");'
    )
    driver$close()

    splittedPriceElements <- unlist(strsplit(priceElement$result$value, "@"))

    neighborHotelRoomPrices <- lapply(splittedPriceElements, function(aElement){
      ## Great Deal\n$80    Deal 5%\n$90
      roomPrice <- unlist(strsplit(aElement,"\n"))
      if(length(roomPrice) > 1){
        return(roomPrice[[2]])
      }
      return (aElement)
    })
    output$result <- renderTable(as.data.frame(unlist(neighborHotelRoomPrices)))
  })
}

shinyApp(ui, server)

Now, I want to extract second page prices as like the screenshot below:

Screenshot_368

Question: How can I extract second page prices in asynchronous way?

Note: Here I extract prices from a single page url and data updated in the particular results section only.

Best Regards, SaleForecast