search

digitalmethodsinitiative / zeeschuimer

A browser extension to collect social media data with.
Other
147 stars 11 forks source link

LinkedIn search does not capture posts #36

Open diana-tircomnicu opened 4 weeks ago

diana-tircomnicu commented 4 weeks ago

When attempting to capture from LinkedIn, I can do so for posts on my feed. However, when I search a hashtag and select posts, there is no file created containing the posts identified in LinkedIn search. Can this be fixed?

dale-wahl commented 3 weeks ago

Which version of Zeeschuimer are you using? Lastest is 1.10.1 which should be visible on the main tool page.

Zeeschuimer captures LinkedIn posts (not groups, jobs, people, etc.). I just tested and it is collecting when I search for "artificial intelligence". It first collects the few posts on the search page, then collects the rest when I click "see all post results". Hashtags work similarly.

diana-tircomnicu commented 3 weeks ago

Hi Dale. I have reinstalled the plug in and retried but I get the same result. It can see posts on the main feed and collects the info while scrolling, but search results are not collected. Would it help if I sent you any screenshots?

dale-wahl commented 3 weeks ago

It could not hurt! Point out to which posts are missed. It also should not recapture a post already in the dataset, if that happens to be the case (e.g., you already collected a post elsewhere).

diana-tircomnicu commented 3 weeks ago

Zeeschuimer

Hi Dale. I have added an image with screenshots and explaining the process. I capture the feed, then I search for a #. Upon looking through search results, it does not record the few posts displayed. When going into post results only, it does not record again. I have tried starting to capture from post search results only, with the same outcome: there were 0 posts captured.

dale-wahl commented 3 weeks ago

Thank you for the detailed response. Frustratingly, I can follow those steps and collected posts. My best guess at the moment is that LinkedIn has changed their site but it is not rolled out everywhere (the only visual difference I see is that my "On this page" is ordered differently). I will try a VPN and see if I can reproduce your issue and get back to you.

dale-wahl commented 3 weeks ago

OK, I think I can recreate it, but only on the occasion. In the meantime, try refreshing the page after you have searched; that seemed to collect for me. Will update you when we find a fix.

dale-wahl commented 3 weeks ago

Ok, well, I thought I was missing posts, but it turns out that LinkedIn will "lazy load" posts that it has already seen/shown in your current session. So when I would search again after deleting the collected posts they would not be collected (because they are already loaded). You can forcing a reload via Ctrl+F5 or Shift+Command+R and they are collected for me. I am not sure if that is your issue or if you are just encountering a different version of their site that I do not have access to troubleshoot.

diana-tircomnicu commented 2 weeks ago

I do not think this is my problem unfortunately. I have tried various methods for capturing the posts after search and it does not work. I'll keep an eye on here to see if it ends up being solved. Thanks for your support.

cringefauna commented 2 weeks ago

I have the same problem when scrolling posts on specific profiles. I'm located in EU if that add any value to know. I have tried hard reloading, opening directly from the hyperlink in extension site... If i try a VPN, do you have any suggestions on where to set it, to not encounter problems with potential changes to LinkedIn?

dale-wahl commented 2 weeks ago

It was working in Amsterdam as of yesterday (and via VPN to the UK). They are likely doing some A/B testing of different versions, but that's just a guess. We will keep checking and see if we cannot identify the issue.

cringefauna commented 1 week ago

thank you so much. Ehe, it is frustrating to know that it works for you, making me doubt if i do it correctly. But thank you, hoping for an update at some point :)