AgiMaulana / Instagram-Comments-Scraper

Instagram comment scraper using python and selenium. Save the comments into excel.
MIT License
145 stars 42 forks source link

Unable to locate element: {"method":"css selector","selector":".MGdpg > button:nth-child(1)"} #7

Open gryff530 opened 3 years ago

gryff530 commented 3 years ago

When I run the code after a full set up it consistently states "Message: no such element: Unable to locate element: {"method":"css selector","selector":".MGdpg > button:nth-child(1)"}" Has this issue come up before? If so, is there a simple solve? I am afraid IG may have changed the CSS identifier for the load_more_comments button

AgiMaulana commented 3 years ago

the code still works on me. Are you really sure all the comments scroll have no ended up so the load more button won't appear?

gugahb commented 3 years ago

I'm having the same problem when clicking "load_more_comment"

image

I installed everything from the beginning and I have the newest version of chrome

I disabled the automatic closing of the browser and realized that it tries to load more comments, but it is only loading image

gugahb commented 3 years ago

the code still works on me. Are you really sure all the comments scroll have no ended up so the load more button won't appear?

whats your version in chromedrive ?

AgiMaulana commented 3 years ago

Sorry for the late response. I just have tried with version 87.0.4280.88 and it works fine. I guess the page you open has not load more button, in another word, all comments have shown in a single load

AgiMaulana commented 3 years ago

Ahh my bad, this is might be a connectivity issue, this script will wait for 1.5 seconds roughly, then will try to click the next, you can try to change the time.sleep(1.5) at this line to desired waiting time in second

smycek commented 3 years ago

Hey @AgiMaulana I experience the same problem as @gugahb and @gryff530. While the script starts correctly, it aborts with an error message after the first attempt to load more comments:

Found <selenium.webdriver.remote.webelement.WebElement (session="dedebb2733ffba0efd1fd79dd74ca3a8", element="8e712b55-2377-48c1-aa90-8f51c0f756e4")>

Message: no such element: Unable to locate element: {"method":"css selector","selector":".MGdpg > button:nth-child(1)"} (Session info: chrome=87.0.4280.88)

I have tried both posts with few comments (less than 20) and with many comments (more than 1000) and i always observe the same behavior.

I also tried Firefox/geckodriver as selenium webdriver for testing purposes. Even changing the value of time.sleep(1.5) did not change the behavior.

Is there any chance that Instagram changed the way comments are loaded?

AgiMaulana commented 3 years ago

where have you located guys? if you use VPN let me know the location did you used, I'll try to check is Instagram has changed the page structure in certain locations.

smycek commented 3 years ago

where have you located guys? if you use VPN let me know the location did you used, I'll try to check is Instagram has changed the page structure in certain locations.

Thanks a lot for your answer! I m based in Germany and have not used a VPN to access Instagram.

AgiMaulana commented 3 years ago

I found out that in some locations Instagram redirects the posting page to login page

AgiMaulana commented 3 years ago

Hi, guys, I am trying to add a login feature but need volunteer developer to try the feature since I haven't found free VPN service for Linux/Ubuntu, you can check the feature is under development in this branch

miguelneto10 commented 3 years ago

hi all, i am facing this issue as well, did anyone solve it ? sometimes it finds the selector .MGdpg but most of time it doesn't find.

MargretWG commented 3 years ago

One hacky way to work around this is to set the sleep before starting scraping to longer enough (30s) in scraper.py line7 so that you have time to manually login and then the scraping will work fine