sandro-salles commented 4 years ago

While testing here I found that sometimes, depending on the amount of opened chats a specific whatsapp account has, the initial load time after a successful qrcode scan (or even after login with previous session data available) can take longer (the page must parse/render all chats before the conditions of auth/isInsideChat are met).

By observing the requests that are triggered from whatsapp web, I was able to find the first requests that indicate that user is authenticated. And they happen several seconds before isInsideChat returns true. Sometimes 10 seconds before. The first one is to this javascript:

https://web.whatsapp.com/lazy_loaded_high_priority_components~lazy_loaded_low_priority_components.fcf172f23f363a24f1bf.js

Then I added this logic and saw a big improvement on the authentication check time:

On browser.ts/initClient:

export async function initClient(
  sessionId?: string,
  config?: ConfigObject,
  customUserAgent?: string
) {
  if (config?.useStealth) puppeteer.use(StealthPlugin());
  browser = await initBrowser(sessionId, config);
  const waPage = await getWAPage(browser);
  if (config?.proxyServerCredentials) {
    await waPage.authenticate(config.proxyServerCredentials);
  }
  await waPage.setUserAgent(customUserAgent || useragent);
  await waPage.setViewport({
    width,
    height,
    deviceScaleFactor: 1,
  });
  const cacheEnabled = config?.cacheEnabled === false ? false : true;
  const blockCrashLogs = config?.blockCrashLogs === false ? false : true;
  const blockAssets = config?.blockAssets || false;
  const interceptAuthentication = !(config?.safeMode);
  await waPage.setBypassCSP(config?.bypassCSP || false);
  await waPage.setCacheEnabled(cacheEnabled);

  if (interceptAuthentication || blockAssets || blockCrashLogs) {
    await(waPage as any)._client.send('Network.enable');

    let patterns = [];
    let authCompleteEv;

    if (interceptAuthentication) {
      authCompleteEv = new EvEmitter(sessionId, 'authComplete');
      patterns.push({ urlPattern: '*_priority_components*' });
    }

    if (blockCrashLogs) patterns.push({ urlPattern: '*crashlogs*' });

    if (blockAssets) {
      (waPage as any)._client.send('Network.setBypassServiceWorker', {
        bypass: true,
      });

      patterns = [
        ...patterns,
        ...[
          { urlPattern: '*.css' },
          { urlPattern: '*.jpg' },
          { urlPattern: '*.jpg*' },
          { urlPattern: '*.jpeg' },
          { urlPattern: '*.jpeg*' },
          { urlPattern: '*.webp' },
          { urlPattern: '*.png' },
          { urlPattern: '*.mp3' },
          { urlPattern: '*.svg' },
          { urlPattern: '*.woff' },
          { urlPattern: '*.pdf' },
          { urlPattern: '*.zip' }
        ],
      ];
    }

    await(waPage as any)._client.send('Network.setRequestInterception', {
      patterns,
    });

    (waPage as any)._client.on(
      'Network.requestIntercepted',
      async ({ interceptionId, request }) => {
        const extensions = [
          '.css',
          '.jpg',
          '.jpeg',
          '.webp',
          '.mp3',
          '.png',
          '.svg',
          '.woff',
          '.pdf',
          '.zip',
        ];

        const req_extension = path.extname(request.url);

        if (
          (blockAssets && extensions.includes(req_extension)) ||
          request.url.includes('.jpg') ||
          (blockCrashLogs && request.url.includes('crashlogs'))
        ) {
          await (waPage as any)._client.send(
            'Network.continueInterceptedRequest',
            {
              interceptionId,
              rawResponse: '',
            }
          );
        } else {
          await (waPage as any)._client.send(
            'Network.continueInterceptedRequest',
            {
              interceptionId,
            }
          );

          if (
            interceptAuthentication &&
            request.url.includes('_priority_components')
          ) {
            authCompleteEv.emit(true);
            waPage.evaluate('window.WA_AUTHENTICATED=true;');
          }
        }
      }
    );
  }

And then, on auth.ts/isInsideChat:

export const isInsideChat = (waPage: puppeteer.Page) => {
  return from(
    waPage
      .waitForFunction(
        "!!window.WA_AUTHENTICATED || (document.getElementsByClassName('app')[0] && document.getElementsByClassName('app')[0].attributes && !!document.getElementsByClassName('app')[0].attributes.tabindex) || (document.getElementsByClassName('two')[0] && document.getElementsByClassName('two')[0].attributes && !!document.getElementsByClassName('two')[0].attributes.tabindex)",
        { timeout: 0 }
      )
      .then(() => true)
  );
};

On auth.ts/retrieveQR:

export async function retrieveQR(waPage: puppeteer.Page, sessionId?:string, autoRefresh:boolean=false,throwErrorOnTosBlock:boolean=false, qrLogSkip: boolean = false) {

  let keepTrying = true

  ev.on('authComplete.**', (isAuthenticated, sessionId) => (keepTrying = false));

  const qrEv = new EvEmitter(sessionId, 'qr');

  if (autoRefresh) {
    const evalResult = await checkIfCanAutoRefresh(waPage)
    if (evalResult === false) {
      console.log('Seems as though you have been TOS_BLOCKed, unable to refresh QR Code. Please see https://github.com/open-wa/wa-automate-nodejs#best-practice for information on how to prevent this from happeing. You will most likely not get a QR Code');
      if (throwErrorOnTosBlock) throw new Error('TOSBLOCK');
    }
  }
  let targetElementFound;
  while (!targetElementFound && keepTrying) {
    try {
      targetElementFound = await waPage.waitForSelector(
        "canvas[aria-label='Scan me!']",
        {
          timeout: 1000,
          visible: true,
        }
      );
    } catch (error) {}
  }

  if (!keepTrying) return true;

  let qrData;
  while(!qrData){
    qrData = await waPage.evaluate(`document.querySelector("canvas[aria-label='Scan me!']")?document.querySelector("canvas[aria-label='Scan me!']").parentElement.getAttribute("data-ref"):false`);
  }
  const qrCode = await waPage.evaluate(
    `document.querySelector("canvas[aria-label='Scan me!']").toDataURL('image/jpeg', 0.6)`
  );
  qrEv.emit(qrCode);
  if(!qrLogSkip) qrcode.generate(qrData,{small: true});
  return true;
}

And finally on initializer.ts/create:

try {
    qrDelayTimeout = undefined;
    shouldLoop = true;

    ev.on('authComplete.**', (isAuthenticated, sessionId) => shouldLoop = false);

    spinner.start('Initializing WA');

   ....

Now authentication resolves faster (specially when sessionData is available, sometimes in less than 3 secs). Maybe you could add that check when safeMode is false or something... what you think?

Regards

smashah commented 4 years ago

Thanks @sandro-salles I'll test this out.

smashah commented 4 years ago

@github-actions run

⚡ Release! ⚡

```js (async () => { function exec(cmd) { console.log(execSync(cmd).toString()); } // Config const gitUserEmail = "github-actions[bot]@users.noreply.github.com"; const gitUserName = "github-actions[bot]"; exec(`echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc`); exec(`git config --global user.email "${gitUserEmail}"`); exec(`git config --global user.name "${gitUserName}"`); exec(`npm i`); exec(`npm run release-ci`); //comment on the issue var result = execSync(`npx auto-changelog -o ./tempchangelog.txt --commit-limit false --template ./compact-keepachangelog.hbs --stdout`).toString(); await postComment(result); //create changelog image exec(`npm run release-image`); exec(`git commit -a -m 'updated release-image'`); exec(`git push --force`); })(); ```

smashah commented 4 years ago

Changelog

🚀 Release 1.9.101 (2020-07-18)

Release 1.9.101 7d5b1c2
Feature: Implemented blockAssets for better launch performance #638 b54a10e
Feature: Implemented performance improvements when safeMode: false #640 c7f697b
API Explorer links on npx bin now clickable b5a09d2
Feature: added LAUNCH_TIME_MS to debugInfo for perf monitoring d5569dd
enhancement: Encourage use of executablePath over useChrome for better launch performance 10b8aa4

smashah commented 4 years ago

This feature is now implemented. In order to use these performance, enhancements please set safeMode: false in config.

Thanks @sandro-salles great work 👍

open-wa / wa-automate-nodejs

Feature Request: Improve isAuthenticated load time (Optionally) with request interceptor #640

Changelog

🚀 Release 1.9.101 (2020-07-18)