Doesn't seem to work on Docker

[microworlds]

Excellent job @oliverdunk 💪

Everything runs great locally. However, when I run it on Docker (Chromium 104, with Xvfb enabled and headful mode), it doesn't load the pop-up; it hangs on this line with the error message:

failed: timeout 30000ms exceeded

Increasing the timeout doesn't help, either. It seems that browser.waitForTarget doesn't get the extension pop-up window in Docker. Any way to get this hack run on Docker, too? 😅

Thanks!

[oliverdunk]

Hey! Would you be able to share the Dockerfile that you're using to setup Chrome and xvfb?

[microworlds]

Thank you, @oliverdunk. Here's it 👇. However, I am running it as an actor on Apify, so I made some modifications to the code. But if you can help get any Docker config that would work to have the extension popup in a Docker env, I'll appreciate that. 😅

FROM apify/actor-node-puppeteer-chrome:18

COPY package*.json ./

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
# ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser

RUN npm --quiet set progress=false \
    && npm install --only=prod --no-optional \
    && echo "Installed NPM packages:" \
    && (npm list || true) \
    && echo "Node.js version:" \
    && node --version \
    && echo "NPM version:" \
    && npm --version

COPY . ./

USER root
RUN DEBIAN_FRONTEND=noninteractive apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y xvfb \
    && rm -rf /var/lib/apt/lists/* \
    && rm -rf /src/*.deb \
    && mkdir -p /tmp/.X11-unix \
    && chmod -R 1777 /tmp/.X11-unix \
    && apt-get update \
    && apt-get install chromium -y

# Run everything after as non-privileged user.
USER myuser

Link to Apify docker image - https://hub.docker.com/layers/apify/actor-node-puppeteer-chrome/latest/images/sha256-a18014d00f5354704ca81a7277b7a1d38bc09346827c6a4417df3eae4a5502cc?context=explore

[oliverdunk]

It looks like there are a few issues here:

• In the latest versions of Chrome, popups are loaded with type "page" rather than "other". This is actually super exciting, as it gets things closer to being usable with Puppeteer out of the box. That does mean that the waitForTarget condition has to change though. I suspect this relates to the work in https://chromium-review.googlesource.com/c/chromium/src/+/4245598.
• In more recent versions of Puppeteer, private fields are more heavily enforced. This means that the hack of overriding _isPageTargetCallback can fail.
• I was slightly lying in the first bullet. It gets loaded with type "other" and then becomes type "page" once it loads (maybe a bug?). This means it gets wrongly created with the OtherTarget type rather than PageTarget and page() returns null.

Here's an updated version of openPopup to use in utils.ts which resolves these issues (tested with Puppeteer 20.5.0):

export async function openPopup(
  browser: puppeteer.Browser,
  worker: puppeteer.WebWorker,
  path: string
): Promise<puppeteer.Page> {
  const extensionId = getExtensionId(worker.url());
  await worker.evaluate("chrome.action.openPopup();");

  const popup = await browser.waitForTarget(
    (target) =>
      target.type() === "page" &&
      target.url() === `chrome-extension://${extensionId}${path}`
  );

  // popup is an OtherTarget, but we want a PageTarget instead
  const popupInternal = popup as any;
  const pageTarget: puppeteer.Target = new (puppeteer as any).PageTarget(
    popupInternal._getTargetInfo(),
    popupInternal._session(),
    undefined,
    popupInternal._targetManager(),
    popupInternal._sessionFactory(),
    false,
    null,
    undefined
  );

  return pageTarget.page();
}

Also as an FYI, the apify base image already has Chromium and xvfb installed, so this is sufficient:

FROM apify/actor-node-puppeteer-chrome:20

COPY package.json ./
COPY dist ./dist
COPY demo-extension ./demo-extension

Let me know how you get on!

[microworlds]

Thanks for your excellent feedback, @oliverdunk.

I tried this new flow, but for some reason, beyond my comprehension, it doesn't seem to work. The older version of openPopup works pretty well locally, but not on Docker. The newer version, however, doesn't seem to work locally. 🤔

For the Docker image, Chrome (114) doesn't seem to support chrome.action.openPopup(); for extensions (MV3), only Chromium, hence why I added the Chromium package (Apify's base image uses the official Chrome build, and openPopup is not supported there, yet).

The older implementation works 101% correctly on my local machine but not on Docker. Perhaps I can share the whole repo with you and some instructions on how to run/reproduce it? 😅

Thanks!

[oliverdunk]

Feel free to share the repo! You're right that openPopup only works in certain channels right now, but it seemed to be working in the Docker image for me, so I assumed it was running a Chromium build.

[TheGreatSimo]

Works for me

[pinghe]

    headless: false,
    args: [
      "--headless=new",
      "--disable-extensions-except=demo-extension",
      "--load-extension=demo-extension",
    ],

args add "--headless=new", demo-extension not loaded

[microworlds]

@pinghe try this:

 headless: "new",
  args: [
    "--headless=new",
    "--disable-extensions-except=demo-extension",
    "--load-extension=demo-extension",
  ],

[pinghe]

@microworlds thx!
Upgrading puppeteer from 19.2.2 to the latest version, 21.6.0, solved this problem.