berstend / puppeteer-extra

πŸ’― Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.36k stars 736 forks source link

Supporting Google Meet #334

Closed mp3michael closed 3 years ago

mp3michael commented 3 years ago

Am building a robot to login and record video meetings and puppeteer-extra is proving quite valuable!

Unfortunately Google Meet isn't fooled. Any connection using Google Meet is identified as a bot and blocked.

Anyone have experience with this or have a workaround?

evading-bot-detection commented 3 years ago

this is interesting. Are you logging into a Google account beforehand?

mp3michael commented 3 years ago

No we're not. We building a service for the public and so we are using an unauthenticated web interface to connect to the meeting, record it and then send the user a video.

We're wanting to launch with Zoom, Meet and Teams but it seems Meet is very wise to Puppeteer. Even with your plugin it detects Puppeteer.

mp3michael commented 3 years ago

I'll get this info and add it to this thread. Standby...

s-artemenko commented 3 years ago

Steps to reproduce:

  1. Create a meeting on meet.google.com, get the meeting link (in my case https://meet.google.com/ypd-hohe-vhs)

  2. Start incognito Chrome or Chromium and use the link to join the meeting:

chrome

  1. Run puppeteer:

const puppeteer = require('puppeteer-extra'); const StealthPlugin = require('puppeteer-extra-plugin-stealth'); puppeteer.use(StealthPlugin()); var browser = await puppeteer.launch({headless: false, args : ['--incognito', '--disable-features=IsolateOrigins,site-per-process', '--disable-infobars', '--no-sandbox', '--disable-setuid-sandbox', ] }); var page = (await browser.pages())[0]; await page.goto('https://meet.google.com/ypd-hohe-vhs', { waitUntil: "networkidle0", timeout: 120000 });



"You can't join this video call" is shown.
In the log you can see that POST request "ResolveMeetingSpace" is failed.

![puppeteer](https://user-images.githubusercontent.com/71767222/94009856-570b2200-fda5-11ea-9ca5-7edca0a9492f.png)
s-artemenko commented 3 years ago

ΠŸΡ€ΠΈΠ²Π΅Ρ‚. Пиши ΠΏΠΎ английски ΠΏΠ»ΠΈΠ·, mp3michael Π½Π΅ ΠΏΠΎΠ½ΠΈΠΌΠ°Π΅Ρ‚, ΠΎΠ½ слСдит. I didn't try with Playwright.

s-artemenko commented 3 years ago

Thank you. We are using some Chrome extension, therefore it will be difficult to switch to Firefox. Do you see any way to make it works on puppeteer + Chrome/Chromium?

s-artemenko commented 3 years ago

Thank you. We are using some Chrome extension, therefore it will be difficult to switch to Firefox. Do you see any way to make it works on puppeteer + Chrome/Chromium?

I'd highly recommend this option enabled await page.setBypassCSP(true)

When you load Meet, there are a bunch of errors in console. Trackdown the exact reason. There is a bug in 2.6.1, so try to downdtrade to the previous versions, like 2.6.0

If I could login using Firefox with obvious fingerprint, and enable videos, then there is no protection in place, they do not track bots.

Ok, thank you. I will try.

berstend commented 3 years ago

Closing until there's more technical info provided.

namnm commented 2 years ago

Having the same issue, not sure how to get pass this

steinathan commented 2 years ago

You can try to use this stealth plugin

https://www.npmjs.com/package/puppeteer-extra

thearnica commented 1 year ago

Confirmed to work with Stealth plugin

yehorpenshyn commented 6 months ago

Unfortunately, I'm having the same issue. I use puppeteer-extra library to launch the browser and page, but I already have had this label 'You can't join this video call', before joining. Therefore a few days ago everything works correctly.

Also I use stealth plugin

siddharthbulia commented 6 months ago

Not working for me either. Using stealth plugin.

Hassaan11 commented 5 months ago

Its works if I logs into google account before joining the meeting but without google login its not working for me either.

akumar044 commented 4 months ago

Has anyone figured out a solution for this? @Hassaan11 @siddharthbulia @yehorpenshyn

So far I have used the StealthPlugin and removed these evasions

stealthPlugin.enabledEvasions.delete("iframe.contentWindow"); stealthPlugin.enabledEvasions.delete("media.codecs");

Nothing seems to be working. Before joining the meeting I get "You can't join this video call".

yehorpenshyn commented 4 months ago

@akumar044 which version of puppeteer do u use?

I had the same error earlier and I found the issue in puppeteer versioning after my upgrading puppeteer package. Just downgrade the version and everything is working perfectly.

akumar044 commented 4 months ago

I'm using these versions @yehorpenshyn

"puppeteer": "22.4.1", "puppeteer-extra": "^3.3.4", "puppeteer-extra-plugin-stealth": "^2.11.1"

yehorpenshyn commented 4 months ago

@akumar044 try to use my list of versions

"puppeteer": "^20.9.0", "puppeteer-extra": "^3.3.6", "puppeteer-extra-plugin-stealth": "^2.11.2",
akumar044 commented 4 months ago

Awesome, it worked!. I'm using "puppeteer-stream": "3.0.9" and the above 3 versions are compatible with it.

Thankyou @yehorpenshyn

Hassaan11 commented 4 months ago

const { executablePath } = require('puppeteer')

I am using "puppeteer": "^19.6.3", "puppeteer-extra": "^3.3.4", "puppeteer-extra-plugin-stealth": "^2.11.1",

this.browser = await launch(puppeteer, { // executablePath: '/usr/local/bin/chromium', executablePath: executablePath(), devtools: false, headless: false, defaultViewport: null, args: [ '--no-sandbox', '--headless=new', '--disable-setuid-sandbox', '--ignore-certificate-errors', '--disable-dev-shm-usage', '--enable-features=ChromeBrowserCloudManagement', '--disabled-setupid-sandbox', ], });

  its working fine if i use 
  `executablePath: executablePath(),`
  puppeteer joins the meeting without asking to login

  but if i try to use custom chromium path like 
  // executablePath: '/usr/local/bin/chromium',
  then it wont work it gives 403 error.
tamagoeggo commented 4 months ago

If you didn’t want to deal with the complexities of implementing this yourself, an alternative is to use Recall.ai for your meeting bots instead. It’s a simple 3rd party API that lets you use meeting bots to get raw audio/video from meetings without you needing to spend months to build, scale and maintain these bots.

Disclaimer: I’m one of the founders.

princemishra01 commented 3 months ago

rtunately Google Meet isn't fooled. Any conn

Hi @akumar044 We were trying the same thing, to get the recorded stream through puppeteer-extra and puppeteer-extra-plugin-stealth, but we are still not able to bypass the google authentication and getting the message "you can't join this video call". By the above discussion I came to know that you have resolved this issue using the different version of above mentioned packages, but we are still not able to resolve it. Here is our code approach we followed to tackle this

const { launch, getStream } = require("../dist/PuppeteerStream");
const fs = require("fs");
const { executablePath } = require("puppeteer");

// make sure to install puppeteer-extra & puppeteer-extra-plugin-stealth to use this example
const puppeteer = require("puppeteer-extra");
const StealthPlugin = require("puppeteer-extra-plugin-stealth");

puppeteer.use(StealthPlugin());

const file = fs.createWriteStream(__dirname + "/test.webm");

async function test() {
  const browser = await launch(puppeteer, {
    executablePath: executablePath(),
    defaultViewport: {
      width: 1920,
      height: 1080,
    },
  });

  const page = await browser.newPage();
  await page.goto("https://meet.google.com/yto-rnuz-ink");
  const stream = await getStream(page, { audio: true, video: true });
  console.log("recording");

  stream.pipe(file);
  setTimeout(async () => {
    stream.destroy();
    await page.close();
    await browser.close();
    file.close();
    console.log("finished");
  }, 1000 * 10);
}

test();

Can you please be able to guide us through this to resolve the issue?

Thanks in advance

akumar044 commented 3 months ago

Hi @princemishra01, I faced the same issue sometime back. I believe adding this should work

const stealthPlugin = StealthPlugin(); stealthPlugin.enabledEvasions.delete("iframe.contentWindow"); stealthPlugin.enabledEvasions.delete("media.codecs"); puppeteer.use(stealthPlugin);

Let me know how it goes!

Spenquatch commented 1 month ago

Using the suggestions here I have been able to get google meet to let me join locally on my PC, but when I try it in a Docker container it always gives me the "You can't join this video call" page. Any suggestions?

buddhikajay commented 1 month ago

I've created a sample project here. @Spenquatch inside docker, I can make it work when I run xvfb-run --auto-servernum --server-args='-screen 0 1280x720x24' npm start but does not work when I add that to CMD. This might be related to this. Please consider raising an MR if you find a solution