Support for Google Tag Assistant (Google Tag Manager preview)

Manduro commented 2 years ago

When using Google Tag Assistant to test a Proxytown implementation of Google Tag Manager, it fails to connect: Screenshot 2022-02-11 at 16 01 04 At first there were some CORS errors for the googletagmanager.com domain. After proxying those I can see the iframe getting added to the page, but it fails to load and some Partytown error occurs:

Screenshot 2022-02-11 at 16 07 01

In the page's <head> I can see the following scripts:

<script type="text/partytown-x" src="http://localhost:3003/~partytownproxy/google-analytics/analytics.js" data-ptsrc="https://www.google-analytics.com/analytics.js" data-ptid="351043767"></script>
<script type="text/partytown-x" src="http://localhost:3003/~partytownproxy/googletagmanager/gtag/js" data-ptsrc="http://www.googletagmanager.com/gtag/js?id=G-44K8TKMTS9&amp;l=dataLayer&amp;cx=c" data-ptid="574473788"></script>
<script type="text/partytown" src="http://localhost:3003/~partytownproxy/googletagmanager/debug/bootstrap" data-ptsrc="https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290" data-ptid="433709864" data-pterror="Error: Error finding instance &quot;1&quot; on window 2 (259405394)
    at sendToMain (http://localhost:3003/~partytown/debug/partytown-ww-sw.js:253:27)
    at queue (http://localhost:3003/~partytown/debug/partytown-ww-sw.js:226:20)
    at callMethod (http://localhost:3003/~partytown/debug/partytown-ww-sw.js:333:20)
    at Node.<anonymous> (http://localhost:3003/~partytown/debug/partytown-ww-sw.js:1338:24)
    at wc (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:53:163)
    at eval (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:66:331)
    at lc (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:47:924)
    at Bc.start (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:65:932)
    at eval (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:70:1364)
    at Proxy.eval (https://www.googletagmanager.com/debug/bootstrap?id=GTM-XXXXXXX&amp;src=GTM&amp;cond=2&amp;gtm=2wg290:77:3)"></script>

And just before the </body>:

<iframe src="about:blank" class="__TAG_ASSISTANT_BADGE" frameborder="0" style="border: 0 !important; bottom: 24px !important; border-radius: 8px !important; box-shadow: 0 3px 5px -1px rgb(0 0 0 / 20%), 0 5px 8px 0 rgb(0 0 0 / 14%), 0 1px 14px 0 rgb(0 0 0 / 12%) !important; clip: initial !important; display: inline !important; height: 200px !important; left: auto !important; margin: 0 !important; max-width: 95% !important; opacity: 1 !important; padding: 0 !important; position: fixed !important; right: 24px !important; top: auto !important; visibility: visible !important; width: 450px !important; z-index: 2147483647 !important;" data-ptwindow="259405394"></iframe>

When switching back to the default GTM implementation (without Partytown), everything works as expected.

orkunzozturk commented 10 months ago

@excelsior091224

I have same issue. I'm using [@astrojs/partytown]

To anyone using Astro or something similar:

Create a gtm.astro component

---
---
<script>
  const searchParams = new URL(window.location.href);
  const isDebug = searchParams.searchParams.get('gtm_debug');

  const scripts = [
      {
          src: 'https://www.googletagmanager.com/gtm.js?id=google-tag-id',
          text: null,
      },
      {
          src: null,
          text: `
      window.dataLayer = window.dataLayer || [];
    function gtag() {
      dataLayer.push(arguments);
    }

    gtag('js', new Date());
    gtag('config', 'GA_MEASUREMENT_ID');
      `,
      },
  ];

  function injectScripts() {
      scripts.forEach((s) => {
          const script = document.createElement('script');
          script.type = isDebug ? 'text/javascript' : 'text/partytown';

          if (s.src) script.src = s.src;
          if (s.text) script.text = s.text;
          document.head.appendChild(script);
      });
  }

  injectScripts();
</script>

proxy the requests in astro.config.mjs

partytown({
  config: {
    resolveUrl: (url, location) => {
      // these were the proxied hosts I wanted to proxy
      const proxiedHosts = [
        'googletagmanager.com',
        'connect.facebook.net',
        'googleads.g.doubleclick.net',
      ]

      if (proxiedHosts.includes(url.hostname)) {
        const proxyUrl = new URL('proxy path', 'proxy origin');
        proxyUrl.searchParams.append('url', url.href);
        return proxyUrl;
      }

      return url;
    },
    forward: ['dataLayer.push', 'fbq'],
  },
}),
``

You can then implement an API endpoint that proxies a request to those `proxiedHosts` or whatever you need.

could anyone please explain the astro.config.mjs part? I don't understand what the proxiedHosts should do. For reference, I'm using Netlify

ethan-ou commented 10 months ago

@orkunzozturk have a look at the Partytown documentation on proxying requests. Since you're using Partytown, you're using a web worker which has a different execution context to your main document. This means you need to specify which url's you want to allow into the web worker.

The ones listed should just work with Google Tag Manager though so you might not need to edit it at all.

On another note, I found I was getting an error in the tag due to this section of the astro config since I didn't replace the proxy path and proxy origin strings:

if (proxiedHosts.includes(url.hostname)) {
  const proxyUrl = new URL('proxy path', 'proxy origin');
  proxyUrl.searchParams.append('url', url.href);
  return proxyUrl;
}

To fix it you can just use the location argument. This should automatically get the origin of your website url:

if (proxiedHosts.includes(url.hostname)) {
  const proxyUrl = new URL(location.origin);
  proxyUrl.searchParams.append('url', url.href);
  return proxyUrl;
}

The full astro config code as before with the fix:

    partytown({
      config: {
        resolveUrl: (url, location) => {
          const proxiedHosts = [
            "googletagmanager.com",
            "connect.facebook.net",
            "googleads.g.doubleclick.net",
          ];

          if (proxiedHosts.includes(url.hostname)) {
            const proxyUrl = new URL(location.origin);
            proxyUrl.searchParams.append("url", url.href);
            return proxyUrl;
          }

          return url;
        },
        forward: ["dataLayer.push", "fbq"],
      },
    }),

tcetin commented 9 months ago

@slawekkolodziej

I played with integrating tag assistant here: https://github.com/slawekkolodziej/partytown/tree/tag-assistant

I wrote a document with what I've found: https://github.com/slawekkolodziej/partytown/blob/tag-assistant/src/lib/web-worker/tag-assistant.md

The implementation is not finished yet. It's rather at early stage / POC. There are some missing parts and the implementation is definitely not production ready.

Is there any progress on it? I tried the script you worked on. It didn't work for me.

geekysaurabh001 commented 9 months ago

Is there any in-depth solution yet for NextJS 12/13? The documentation is not clear tbh.

geekysaurabh001 commented 9 months ago

So, no solution till now. It also looks like an abandoned project. Sigh!

Finally resolved all the CORS error and everything yet GTM container doesn't work. Basically, it's a library not ready for production but hyped through influencers as if it is.

arosesiejdev commented 8 months ago

Running into this issue as well.. Same as @geekysaurabh001

gioboa commented 8 months ago

you can write me on Discord Let's solve this issue 💪

slawekkolodziej commented 8 months ago

I put together my solution in a npm module some time ago. You can check it out here: @superside-oss/partytown-gtm.

There are some things that are still not perfect, but it works okay for us.

marcowuethrich commented 8 months ago

After long hours of back and forth, I was able to set up an Angular 17 app with SSR and i18n. The Google Tag Assistant now works as desired. In case anyone has the same problem, I created a simple example repository: github.com/find-ida/angular-ssr-partytown

gioboa commented 8 months ago

Thanks @marcowuethrich for you commitment. Do you have a public application with this implementation?

marcowuethrich commented 8 months ago

@gioboa Not a public repository but our company landing page uses this implementation find-ida.com. If you have any specify question about it let me know.

gioboa commented 8 months ago

@gioboa Not a public repository but our company landing page uses this implementation find-ida.com. If you have any specify question about it let me know.

That's nice, would you like to add your website here? https://github.com/BuilderIO/partytown/issues/497 Thanks for sharing.

sthobis commented 8 months ago

@marcowuethrich In my opinion disabling partytown by using text/javascript instead of text/partytown is not an ideal solution as it doesn't reflect actual production usage. Thanks for sharing though.

I put together my solution in a npm module some time ago. You can check it out here: @superside-oss/partytown-gtm.

There are some things that are still not perfect, but it works okay for us.

I tried replicating this but I got this error https://github.com/BuilderIO/partytown/issues/202 Do you have some clue on what might be causing this @slawekkolodziej

slawekkolodziej commented 8 months ago

@sthobis I did not face that specific issue.

By "There are some things that are still not perfect, but it works okay for us" I meant two things: a) TagAssistant banner does not load properly. It shows transparent window with that shadow / border. Exactly what I can see in the screenshot in your post). b) I only implemented API for Tag Assistant browser plugin. If the plugin is missing, my approach won't work.

However there are 3 main parts that needs to be taken care of:

Tag Assistant browser plugin (this adds GTM-specific APIs to the window object)
Code that runs in main thread AND worker to work as a communication channel between GTM and that plugin-specific browser API.
Proxying some GTM-specific JS files as they are missing proper CORS headers

yurasytnyk commented 7 months ago

this is what worked for me in Next.js pages router (_document.tsx) sorry for the bad text formatting

import { Partytown } from "@builder.io/partytown/react";
import { Html, Head, Main, NextScript } from "next/document";

export default function Document() {
  return (
    <Html lang="en">
      <Head>
        <Partytown
          debug={true}
          forward={["dataLayer.push"]}
          set={(opts) => {
            const isDebugging = opts?.window?.location?.search.includes("gtm_debug");

            if (isDebugging && opts?.name === "type" && opts?.nodeName === "SCRIPT") {
              return opts.prevent;
            }

            return opts.continue;
          }}
        />

        <script
          id="gtm"
          type="text/partytown"
          dangerouslySetInnerHTML={{
            __html: `(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-id');`,
          }}
        />
      </Head>

      <body>
        <Main />
        <NextScript />

        <noscript>
          <iframe
            src="https://www.googletagmanager.com/ns.html?id=GTM-id"
            height="0"
            width="0"
            className="hidden"
          ></iframe>
        </noscript>
      </body>
    </Html>
  );
}

geekysaurabh001 commented 7 months ago

this is what worked for me in Next.js pages router (_document.tsx) sorry for the bad text formatting

import { Partytown } from "@builder.io/partytown/react";
import { Html, Head, Main, NextScript } from "next/document";

export default function Document() {
  return (
    <Html lang="en">
      <Head>
        <Partytown
          debug={true}
          forward={["dataLayer.push"]}
          set={(opts) => {
            const isDebugging = opts?.window?.location?.search.includes("gtm_debug");

            if (isDebugging && opts?.name === "type" && opts?.nodeName === "SCRIPT") {
              return opts.prevent;
            }

            return opts.continue;
          }}
        />

        <script
          id="gtm"
          type="text/partytown"
          dangerouslySetInnerHTML={{
            __html: `(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-id');`,
          }}
        />
      </Head>

      <body>
        <Main />
        <NextScript />

        <noscript>
          <iframe
            src="https://www.googletagmanager.com/ns.html?id=GTM-id"
            height="0"
            width="0"
            className="hidden"
          ></iframe>
        </noscript>
      </body>
    </Html>
  );
}

Can we test using Preview mode via this method?

yurasytnyk commented 7 months ago

yes you can, I tested it yesterday

geekysaurabh001 commented 7 months ago

yes you can, I tested it yesterday

Doesn't work. Same CORS issue is present again.

yurasytnyk commented 7 months ago

@geekysaurabh001 did you publish your GTM container? It can cause problems with CORS if not published

callum-gander commented 5 months ago

Is there any progress on this?

It feels like a pretty big oversight that scripts like GTM are arguably the biggest and most widely used use case for PartyTown yet the issue of how to run them in PartyTown and debug/preview with Google Tag Assistant isn't properly supported.

A lot of the above solutions seem to basically be "don't run the script with PT when you're debugging" but we can't just assume that everything that works perfectly in the main thread is going to work correctly in the PT worker, as @sthobis has more or less said.

@slawekkolodziej's solution here seems to be a step towards trying to resolve this but has been sitting in this thread for over a year without proper investigation by the maintainers and integration of it into the codebase + docs as a recommendation.

This library attempts to solve a really important problem for a lot of websites and it's great that it trying to. However, the DX is really poor and makes it a massive risk integrating for companies for whom GTM and 3P scripts have important roles in their business.

I really would like to use this library, but the fact that it's been around for 3 years and it's still in beta with issues like this outstanding for over a year makes it really, really difficult, as it just feels wildly under maintained at this point. It makes it hard to persuade the other people on my team that this is something we should use, even if we do definitely need it due to the impact of 3P scripts on the main thread

gioboa commented 5 months ago

@callum-gander what's your stack? There are a lot of different errors in this thread. Which is yours?

callum-gander commented 5 months ago

Next.js 14, PT is successfully loading GTM but Google Tag Assistant can't connect to it on http://localhost:3000, presuming because GTM is running in the service worker, any up to date advice on handling this?

Thanks for the fast response btw

gioboa commented 5 months ago

You can take inspiration from @marcowuethrich code, he added a specific check in his angular application. Here is the code @callum-gander Did you already try did approach?

callum-gander commented 5 months ago

Correct me if I'm wrong, but that solution is the type of solution I said isn't optimal in my previous message.

All that's doing seems to be switching the type="text/partytown" to type="text/javascript" on the GTM scripts when gtm_debug is present in the URL. This isn't ideal as that's just testing whether events are all working on the main thread, which we can assume they are, what we want to test is whether they're still all coming through when GTM is running in the PT worker, which we can't assume before pushing to production. We need to be able to robustly test that everything is working correctly with the partytown worker before we push to production as it can mess up all sorts of experiments that we're running.

This seems to be possible from @slawekkolodziej's work here https://github.com/slawekkolodziej/partytown/blob/tag-assistant/docs/tag-assistant.md but I'm unable to get it running and was posting again here to see if anyone had success with this method or whether the maintainers had implemented something similar to this, as this is the ideal solution for this problem, as it solves the problem of how to connect GTM running in PartyTown to Google Tag Assistant not just running GTM on the main thread

slawekkolodziej commented 5 months ago

@callum-gander, the link you are referring to was my initial work. I ended up wrapping all of that in a slightly more convenient partytown plugin here: https://github.com/superside-oss/partytown-gtm-plugin, did you try using it?

callum-gander commented 5 months ago

Hey @slawekkolodziej, so yes, I haven't gone 100% with your code, but I've broadly used the approach you use within it and used a lot of the code within your repo, which works great, amazing work! Happy to go into detail if people want to know how I did this but I'd strongly recommend anyone having this issue to just use @slawekkolodziej's plugin, the only reason we don't is due to some internal complexities that mean it's better to hand implement

I've broadly got this working except for the issues mentioned below about the TagAssistant banner not loading properly. Any updates on if it's possible to fix this? Or if it causes any further issues? e.g. is it just a minor UI bug and all events will still properly go through?

@sthobis I did not face that specific issue.

By "There are some things that are still not perfect, but it works okay for us" I meant two things: a) TagAssistant banner does not load properly. It shows transparent window with that shadow / border. Exactly what I can see in the screenshot in your post). b) I only implemented API for Tag Assistant browser plugin. If the plugin is missing, my approach won't work.

However there are 3 main parts that needs to be taken care of:

Tag Assistant browser plugin (this adds GTM-specific APIs to the window object)

Code that runs in main thread AND worker to work as a communication channel between GTM and that plugin-specific browser API.

Proxying some GTM-specific JS files as they are missing proper CORS headers

Despite your amazing work reversing all this and getting this working as a separate package, I can't help but feel like this really deserves to be within the main PartyTown repo + docs, not a separate 3P package.

As I raised earlier, GTM is cited as the first use case on the PartyTown docs page here and realistically it's probably the primary reason for adoption for most users, as GTM is so widely used and a complete nightmare on performance. Not having robust handling of debugging GTM with PartyTown (not just turning it off during debugging), both in the library itself and within the documentation as well, is really, really poor DX and feels like a barrier to adoption. Out of interest, why did you set it up as a separate package? Did you raise a PR at all about bringing it into the main repo? Is there any possibility or likelihood of you doing so?

slawekkolodziej commented 5 months ago

Out of interest, why did you set it up as a separate package?

I created bunch of other PRs that made the plugin possible. But I had mixed feelings about creating a separate PR that adds library-specific code to Partytown core. I needed something that I can release & integrate quickly.

But I agree, I would love to have GTM debugging work out of the box when I first tried Partytown.

callum-gander commented 5 months ago

Thanks for your response @slawekkolodziej, that makes sense given the need to just get it working asap. @gioboa you seem to be a frequent contributor, how can we go about getting the ball rolling on integrating this into PartyTown?

Would it be a good idea to add this to the docs as a temporary bandaid, e.g. "use this plugin for debugging GTM for now" and linking @slawekkolodziej's package and then opening a PR to get some of this stuff more natively integrated into the PartyTown library over time? Happy to help with any PRs on this work btw

gioboa commented 5 months ago

The best way to integrate that is:

to reproduce the problem locally
create the code to fix the issue
drop a PR to correct the scenario
double check the resolution of the scenario

I think we can't create an integration test to check this issue inside the GH actions because we need public keys and different tools

techfg commented 4 months ago

@gioboa - Thanks for merging #573 & #493, great to see that fix made! Regarding this issue, I don't believe it should have been closed. Even with the fix in #573, while the situation with GTM should be improved when not using atomics, there are still issues/limitations with GTM with atomics and service workers that this issue mentions. For example, the workaround provided by @slawekkolodziej is still required and even with that, functionality is not 100% supported when PT is used.

maruthasalamr commented 3 months ago

I'm getting CORS Policy error even though create reverse proxy for Hubspot analytics

Partytown version 0.10.1 Gatsby version 5.11.0 Web host Provider - AWS Cloudfront Reverse proxy setup for hubspot analytics in cloudfront

Reverse Proxy setup uin cloudfront

Partytown configuration

Gatsby-ssr file

export const onRenderBody = ({ setHeadComponents, setPreBodyComponents }) => {
  setHeadComponents([
    <Partytown
      key="partytown"
      resolveUrl={(url,location,type) => {       
          if (type === 'script' && url.hostname.includes('analytics')) {
            var proxyUrl = new URL('https://d1gec6izgol1hj.cloudfront.net');
            proxyUrl.searchParams.append('url', url.href);
            return proxyUrl;
          }

          return url;
      }}
      debug={true}
      forward={['dataLayer.push']}
    />,
    <script
      key="plugin-google-tagmanager"
      type="text/partytown"
      dangerouslySetInnerHTML={{
        __html: `(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
          new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
          j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
          'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
          })(window,document,'script','dataLayer','GTM-KXJZBP');`,
      }}
    />,
  ]),
    setPreBodyComponents([
      <noscript
        key="gtm"
        dangerouslySetInnerHTML={{
          __html: `
                  <iframe src="${ORIGIN}/ns.html?id=${gtmTrackingId}" height="0" width="0"
                      style="display:none;visibility:hidden"></iframe>
                `,
        }}
      />,
    ]);
};

someone can you please help me on this ?

callum-gander commented 3 months ago

Probably worth mentioning that I'd gotten @slawekkolodziej's solution working but recently, whether it's due to something on our end, an issue with GTM, some change in chrome or an issue with his solution, GTM Assistant now won't connect to GTM in PartyTown. It's not something I'm capable of debugging or something I can post a codesandbox of to demonstrate the issue, I can only point out that going down the root of integrating partytown-gtm is now not working for me. On connecting to GTM Assistant, I get thousands of Tag Assistant Set receiver logs and GTM Assistant says GTM does not exist on the page.

As much as I wanted to keep pursuing this to get it working again, the lack of a concrete solution within the library itself from the core team for what I'd argue is a pretty core usecase and the length of time this issue has been open, kind of makes it difficult for me to justify to my team that it's worth putting more time into it and we won't be using PartyTown any time soon as a result.

I think the @andre-mr's comment sums it up best here https://github.com/BuilderIO/partytown/issues/554#issuecomment-1966971121. Losing analytics data or issues around being able to guarantee they're working before going to production means that any potential performance gains from PartyTown aren't worth pursuing. In particular the iOS issue at the top of the below issues concerns me

I just thought this might be worth bringing up in case it helps justify additional resources from BuilderIO being put towards issues like this. There's clearly been a lot of amazing work put into this library and it's the only library of it's kind but fixing issues like this one feel core to it's wider adoption. Thanks for all the assistance on this!

jemys89 commented 3 months ago

Hello marcowuethrich I've implemented the same solution you mention in your repository and I'm encountering the same issue mentioned at the end of this thread... Tag Manager's debugging no longer works.

In addition to that, in Angular, navigation fails quite often with Partytown. It's as if the Service Worker is blocking the navigation event... you have to reload the page to navigate properly.

You can validate it here:

If you navigate from the home to the blog, then the cards don't work correctly. https://find-ida.com/de/blogs

It's a great pity, given the wonderful work that has been done by the community with this library, but I'm afraid I'm going to have to uninstall it due to the high number of issues related to something as important as monitoring users/business.

I'm looking forward to a solution.

Many thanks to everyone!

indykoning commented 2 months ago

I'm currently running with tag assistant without issue using https://github.com/rapidez/gtm/pull/23/files

based on https://github.com/superside-oss/partytown-gtm-plugin

BuilderIO / partytown

Support for Google Tag Assistant (Google Tag Manager preview) #72