zaiinabyy commented 2 years ago

hello, is there a way to export the query results to csv file?

jessebehrens commented 2 years ago

Yes! Save it to a dataframe.

First I create a list:

def on_data(data: EventData): job_postings.append([data.job_id, data.location, data.title, data.company, data.date, data.link, data.description])

Then I put that in a DF

Put the job postings into a dataframe

df = pd.DataFrame(job_postings, columns=['Job_ID','Location','Title', 'Company', 'Date', 'Link', 'Description'])

leonpawelzik commented 2 years ago

Where do you place that code? Also, it prompts, that job_postings is not defined.

Prashanthvsdvn commented 1 year ago

Find the function below and add the last line there. Be careful of the tab space. Need to tab once

def on_data(data: EventData):
    print('[ON_DATA]', data.title, data.company, data.company_link, data.date, data.link, data.insights,
          len(data.description))
    job_postings.append([data.job_id, data.location, data.title, data.company, data.date,
data.link, data.description])

and add these at the bottom

job_postings = []
scraper.run(queries)
df = pd.DataFrame(job_postings, columns=['Job_ID','Location','Title', 'Company','Date', 'Link', 'Description'])
df.to_csv("jobs.csv")

spinlud / py-linkedin-jobs-scraper

export results to csv #21

Put the job postings into a dataframe