search

mguideng / gdscrapeR

:package: R package to easily web scrape Glassdoor company reviews. Write up of demo:
https://mguideng.github.io/2019-02-27-scrape-glassdoor-gdscrapeR/
22 stars 8 forks source link

Error in 1:maxResults : argument of length 0 #1

Closed amrrs closed 5 years ago

amrrs commented 5 years ago

Hi,

Thanks for this work. I tried this package but couldn't get the data.

My code 


library(gdscrapeR)

odessa <- gdscrapeR::get_reviews("E273099")

ended up with this error, after i typed 1

Error in 1:maxResults : argument of length 0
mguideng commented 5 years ago

Hi amrrs, I was able to run your code and it returned a dataframe object. odessa.xlsx

A known limitation with scrapers is the need to keep up with the changes to a website's structure. It's possible you got a version of the web page that gdscraper does not recognize. I recommend trying again at later times, until you get a version that it supports. Let me know if you get further issues. Thanks, mguideng

mjahanshahi commented 5 years ago

Hi guideng, I had this working originally but have also developed the same problem (I've been trying every few hours since yesterday afternoon). I looked at the source code and the issue seems to be with the html_nodes selector. When I look at the source html code of glassdoor pages, I can't find the .tightVert.floatLt strong, .margRtSm.margBot.minor element anywhere.

mguideng commented 5 years ago

amrrs - I'm on it and will update.

mguideng commented 5 years ago

I updated the package (v1.1) to address this issue.

wenxin-lab commented 4 years ago

Hi I tried your code, but it keeps saying :

Number of web pages to scrape: 118 Starting P1Error in value[3L] : Could not scrape data from website. Try again later. Exiting function.

do you know how to fix it?

subhroisback commented 4 years ago

Hi Maria, Thanks a lot for this work. When I tried this package but couldn't get the data. get_reviews(companyNum = "E428473") I encountered this error - Error in 1:maxResults : argument of length 0

I think somewhere in this function the error is - purrr::map_dfr(1:maxResults, function(i) { cat(" P", i, sep = "") pg <- xml2::read_html(httr::GET(paste(baseurl, companyNum, "_P", i, sort, sep = "")))

ashlynlowe commented 4 years ago

Hi Marie (@mguideng),

When was the last time the package was updated? It looks like Glassdoor may have changed its html schema and javascript recently. I tried running get_reviews and received the following error:

Error in 1:maxResults : argument of length 0

Thank you!

stevekozel commented 3 years ago

Also getting "Error in 1:maxResults : argument of length 0"