search

haasr / indeed-jobs-searcher

Simple CLI-based tool for extracting and storing relevant job info. from bulk job searches on indeed.com
MIT License
7 stars 2 forks source link

Only pulling first page of results #1

Open shadyKatie opened 1 month ago

shadyKatie commented 1 month ago

It seems that only the first page of results is appearing in the export (~14 jobs). Is there a way to pull all results for a given search?

haasr commented 1 month ago

Good point...

I should fix that. I'm guessing I could just add some logic to check if there is a next page by looking for the x-paths of these buttons at the bottom and then keep clicking them as long as there are more and repeating the scrape.

[cid:b18adc9a-42b4-4e53-a06d-31b98a3764f5]

I'd like to work on this soon but I am hella busy right now. If you figure this out before I get to it, send me a pull request! I'm really shit with x-paths and Selenium but ChatGPT was helpful in re-writing this last time Indeed changed their page structure.

Sincerely,

Ryan Haas • he/him/his Professor of Using Information Technology East Tennessee State University Department of Computing

From: shadyKatie @.> Sent: Tuesday, May 28, 2024 2:34 PM To: haasr/indeed-jobs-searcher @.> Cc: Subscribed @.***> Subject: [haasr/indeed-jobs-searcher] Only pulling first page of results (Issue #1)

It seems that only the first page of results is appearing in the export (~14 jobs). Is there a way to pull all results for a given search?

— Reply to this email directly, view it on GitHubhttps://github.com/haasr/indeed-jobs-searcher/issues/1, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALYCXYDRRGGTIV4BWWCICW3ZETE3RAVCNFSM6AAAAABINO36R6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDCNRYHA3DOMQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>