How to save to file?

[jbdw85]

This works great and I am still learning, but how can this be saved to an .csv file or json? Or is there is good way to copy this from the terminal?

[spinlud]

Hi @jbdw85, have you looked at the examples?

Basically once you have registered a listener for the data event, you can do whatever you want with the data (write to local disk, upload to an S3 bucket, push to a queue...), it is up to you.

Here is a small example for appending data to a local file:

const fs = require('fs').promises;
const { LinkedinScraper, events } = require('linkedin-jobs-scraper');

(async () => {
    const scraper = new LinkedinScraper({
        headless: true,
        slowMo: 10,
    });

    // Add listeners for scraper events    
    scraper.on(events.scraper.data, async (data) => {
        const line = `${data.title}\t${data.date}\t${data.description.length}\n`;
        console.log(line);
        await fs.appendFile('output.txt', line, { flag: 'a' });
    });

    scraper.on(events.scraper.error, (err) => { console.error(err); });
    scraper.on(events.scraper.end, () => { console.log('\nE N D (ツ)_.\\m/') });

    // Custom function executed on browser side to extract job description
    const descriptionProcessor = () => (document.querySelector(".description__text"))
        .innerText
        .replace(/[\s\n\r]+/g, " ")
        .trim();

    await Promise.all([
        scraper.run(
            'Node.js',
            'United Kingdom',
            {
                paginationMax: 1,
                descriptionProcessor,
                optimize: true
            }
        ),
    ]);

    await scraper.close();
})();

[jbdw85]

amazing, thank you!