Continuing a session

[dr-nyt]

Hey!

I have a question rather than a bug. So I see that when I connect to a Cloudflare protected site for the first time, it takes the time needed for verification, and the next time during the same session if I were to make a request to the same site, it would go through instantly.

My question is that can I somehow save the cookie used for the session and use it again when I restart the session.

I'm fairly new to all this so I'm not very familiar with web dev.

[ghost]

Sup @dr-nyt,

You'll want to keep a reference to the cookie jar that cloudscraper uses. The cookie jar is a tough.CookieJar. You can import tough-cookie and create a jar or use the convenience method i.e. const jar = cloudscraper.jar(). If you pass that jar to cloudscraper, all of the cookies(aka the session) will be stored in it.

// Same usage as https://github.com/request/request
const cloudscraper = require('cloudscraper').defaults({ jar });

Now that you have a reference to the jar that will be used to store cookies, you may persist that data to the filesystem. There are modules on npm to help with that: https://www.npmjs.com/package/tough-cookie-file-store

This is a modified version of the tough-cookie-file-store example code: I only replaced request with cloudscraper.

var cookieStore = require('tough-cookie-file-store');
var CookieJar = require('tough-cookie').CookieJar;
var jar = new CookieJar(new cookieStore('./cookie.json'));

/* check if cookie is empty or expired */
var cookieStore = require('tough-cookie-file-store');
var cookieInstance = new cookieStore('./cookie.json');
cookieInstance.isExpired() // will return True if the cookie is expired
cookieInstance.isEmpty() // will return True if cookie is empty

/* request example */
var cookieStore = require('tough-cookie-file-store');
var j = cloudscraper.jar(new cookieStore('./cookie.json'));
cloudscraper = cloudscraper.defaults({ jar : j })
cloudscraper('http://www.google.com', function() {
  cloudscraper('http://images.google.com')
})

cheers

[ghost]

I almost forgot to mention that, due to UA randomization, you'll want to persist the request headers as well. So that you use the same UA/headers/cookies combination when continuing the session.

Save the headers:

const cloudscraper = require('cloudscraper');
const fs = require('fs');

const json = JSON.stringify(cloudscraper.defaultParams.headers);
fs.writeFileSync('./headers.json', json, 'utf-8');

Load the headers:

const cloudscraper = require('cloudscraper').defaults({ headers: require('./headers') });

[dr-nyt]

Thank you, that helps alot! So should I store the headers once on launch and keep using those indefinitely?

[ghost]

@dr-nyt Yup and Yw :)