phantombuster / nickjs

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
https://nickjs.org
ISC License
500 stars 48 forks source link

Trying to save content with fs.writeFile #60

Closed dgateles closed 5 years ago

dgateles commented 5 years ago

Im trying to save a json generated but the fs.writeFile step is skipped Here is my code:

process.env.NICKJS_PROXY = "";
process.env.http_proxy = "";
const url = 'https://fontawesome.com/cheatsheet';
const fs = require('fs');
const Nick = require("nickjs")
const nick = new Nick({
  headless: false,
})

;
(async () => {
  const tab = await nick.newTab()
  await tab.open(url)
  await tab.untilVisible("main") // Make sure we have loaded the page
  const iconsList = await tab.evaluate((arg, callback) => {
    // Here we're in the page context. It's like being in your browser's inspector tool
    var groups = {};
    var sections = window.document.getElementsByClassName('cheatsheet-set');
    for (const section of sections) {
      const names = [];
      groups[section.id] = names;
      var icons = section.getElementsByClassName('icon');
      for (const icon of icons) {
        const name = icon.getElementsByTagName('dd')[0].innerText;
        names.push(name);
      }

    };
    callback(null, groups)
  })
  await fs.writeFile('icons.json', JSON.stringify(iconsList, null, 2), (err) => {
    if (err) throw err;
    console.log('The file has been saved!');
  });
})()
.then(() => {
    console.log("Job done!")
    nick.exit()
  })
  .catch((err) => {
    console.log(`Something went wrong: ${err}`)
    nick.exit(1)
  })
TASnomad commented 5 years ago

Hello @dgateles,

fs.writeFile is an asynchronous function that why it skipped during the execution of your script (writeFile doc).

You should try the synchronous version of this function fs.writeFileSync (writeFileSync doc) instead.

Best

dgateles commented 5 years ago

Worked now!