GADS_SSI error

[jbrs360]

I've configured a 1PD Scheduler through the interface using GADS SSI.

• Data source: Big Query
• Data destination: Google Ads SSI

There is access to both platforms as I've used the same credentials/configurations in a Python project before trying with Tightlock.

To limit errors:

• I've validated my BigQuery data to what the code expects (Data schema). I've tried with limited columns and limited data to limit errors
• I've run multiple tests with different data sets, and different data types.

When running the 1PD Scheduler connection I get 1 error: "INVALID_ARGUMENT" Count: 1. I have no idea how to proceed. I've narrowed everything down where I can't change anything anymore and the error message doesn't help me.

[caiotomazelli]

Hi @jbrs360, sorry for the delay coming back to you.

Can you provide the smallest set of columns that allow you to reproduce this error?

I will investigate the issue and get back to you by the end of this week.

[jbrs360]

Sure, no problem. I've attached the schema and a test row from the dummy data. The customer_id and conversion_action_id is however correct for the clients DK market account.

[image: image.png]

[image: image.png]

On Tue, 4 Jun 2024 at 15:23, Caio Tomazelli @.***> wrote:

Hi @jbrs360 https://github.com/jbrs360, sorry for the delay coming back to you.

Can you provide the smallest set of columns that allow you to reproduce this error?

I will investigate the issue and get back to you by the end of this week.

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2147528884, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVNZB2HQDD6ZTMFGT7RTZFW5W3AVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBXGUZDQOBYGQ . You are receiving this because you were mentioned.Message ID: @.***>

[caiotomazelli]

Hi Jacob, I was able to reproduce.

In my case, the error was related to SSI not being allowed in the target account:

>, <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.INVALID_ARGUMENT
    details = "Request contains an invalid argument."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:173.194.212.95:443 {grpc_message:"Request contains an invalid argument.", grpc_status:3, created_time:"2024-06-04T16:35:47.152425085+00:00"}"
>, errors {
  error_code {
    offline_user_data_job_error: NOT_ON_ALLOWLIST_FOR_STORE_SALES_DIRECT
  }

Notice how Tightlock was exposing only the StatusCode ("invalid argument"). I sent PR #169 to better expose the error message.

As SSI requires a manual register step to be allowed, make sure you followed these steps for the target account: https://support.google.com/google-ads/answer/9995886?sjid=9733441823520280530-SA#eligibility

In case your account is already allow-listed, you can retry with the new version (once the PR mentioned above is merged) to see the specific error message you're getting in your case.

Hope this helps!

[jbrs360]

Thank you Caio for the swift answer!

Sounds odd. Before we started using Tightlock we tried to build a cloud function with code from Google's developer docs https://developers.google.com/google-ads/api/docs/conversions/upload-store-sales-transactions#code_example and got it to work until we hit some API limits. However, we got it to work successfully with the provided ids.

I've reached out to Grace again just for confirmation. Do you think the error could be of something else?

On Tue, 4 Jun 2024 at 18:44, Caio Tomazelli @.***> wrote:

Hi Jacob, I was able to reproduce.

In my case, the error was related to SSI not being allowed in the target account:

, <_InactiveRpcError of RPC that terminated with: status = StatusCode.INVALID_ARGUMENT details = "Request contains an invalid argument." debug_error_string = "UNKNOWN:Error received from peer ipv4:173.194.212.95:443 {grpc_message:"Request contains an invalid argument.", grpc_status:3, created_time:"2024-06-04T16:35:47.152425085+00:00"}" , errors { error_code { offline_user_data_job_error: NOT_ON_ALLOWLIST_FOR_STORE_SALES_DIRECT }

Notice how Tightlock was exposing only the StatusCode ("invalid argument"). I sent PR #169 https://github.com/google-marketing-solutions/Tightlock/pull/169 to better expose the error message.

As SSI requires a manual register step to be allowed, make sure you followed these steps for the target account: https://support.google.com/google-ads/answer/9995886?sjid=9733441823520280530-SA#eligibility

In case your account is already allow-listed, you can retry with the new version (once the PR mentioned above is merged) to see the specific error message you're getting in your case.

Hope this helps!

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2147976184, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVN5VG6NVSBXU2BPWQZTZFXVHJAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBXHE3TMMJYGQ . You are receiving this because you were mentioned.Message ID: @.***>

[caiotomazelli]

Hey Jacob,

If you already tested before, this might be something else.

I just merged the changes, so the proper error should be exposed now. Please redeploy and test again and feel free to reopen the issue if you still can't figure it out with the improved error message.

[jbrs360]

Hi Caio,

I've started from scratch and created a new connection with a new data source and destination. I'm still getting the INVALID_ARGUMENT error when running the connection.

I don't see any new or improved error message.

[caiotomazelli]

Hi Jacob,

Just confirming: Have you redeployed your Tightlock backend with the latest version (merged on June 5)?

The change we made to the SSI connector (Line 120) should show you a more detailed version. So either your version is not updated or the API is not providing a detailed description of the problem.

Unfortunately I'm not able to debug the problem directly with your account, so I'll wait for your confirmation before investigating further.

[jbrs360]

Hi Caio,

We've redeployed the Tighlock backend and more detailed error messages are now available. We got a bit further but now I'm experiencing "The required field was not present."

Below you'll find some dummy data from our BQ table. All required fields should be present.

[caiotomazelli]

Hey Jacob,

Looks like the only thing that is missing now is at least one user identifier. Even though there is no single identifier that is required, you need to provide at least one of: email, telephone or address.

[jbrs360]

Of course.

I've added multiple but it still fails unfortunately.

[caiotomazelli]

Thanks for confirming,

I'm trying to reproduce and see if I can investigate this in some other way, I'll keep you posted.

In the meantime, if I were to implement a custom debug version to investigate this issue, would you be willing to deploy it? Since I'm struggling to get access to an account that is enabled to use SSI, I might rely on your testing to get this fixed, if you are interested in helping out.

[jbrs360]

Yes, sure, we could make that happen.

On Thu, 20 Jun 2024 at 20:08, Caio Tomazelli @.***> wrote:

Thanks for confirming,

I'm trying to reproduce and see if I can investigate this in some other way, I'll keep you posted.

In the meantime, if I were to implement a custom debug version to investigate this issue, would you be willing to deploy it? Since I'm struggling to get access to an account that is enabled to use SSI, I might rely on your testing to get this fixed, if you are interested in helping out.

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2181257289, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVN4PMW4XAPVWQ2Z5LJDZIMLDNAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBRGI2TOMRYHE . You are receiving this because you were mentioned.Message ID: @.***>

[caiotomazelli]

Hi Jacob,

I was able to get access to an SSI enabled account and I was able to send conversions through with this minimal example:

id,customer_id,conversion_action_id,conversion_date_time,conversion_micro_value,email
1,XXXXX,YYYYYY,2024-06-18 11:00:00-10:00,10000,abc@gmail.com
2,XXXXX,YYYYYY,2024-06-18 11:00:00-11:00,20000,def@gmail.com

I found a bug in the way related to the optional custom_key parameter (PR #175 ).

That said, this parameter is only optional if you have not set it up in your Ads account.

All that being said, can you provide additional information about the error you're getting, since I was not able to reproduce? If you can also provide redacted information about the setup of your destination (i.e. if you are providing all the required fields in the UI) that would be great.

I'm still trying to find a good way to debug remotely...one option would be for you to temporarily invite me (caiotomazelli@google.com) to your Ads account, so that I can try sending a test hit and see if I can reproduce the problem.

[jbrs360]

Hi Caio,

I unfortunately can't give you access without discussing this with our client.

At the moment we don't receive any error message. Below you will see the latest run together with the first row of the data we're sending. No custom keys are sent with it. I've manually created the schema to match with the requirements. [image: image.png] [image: image.png]

On Fri, 21 Jun 2024 at 20:05, Caio Tomazelli @.***> wrote:

Hi Jacob,

I was able to get access to an SSI enabled account and I was able to send conversions through with this minimal example:

id,customer_id,conversion_action_id,conversion_date_time,conversion_micro_value,email 1,XXXXX,YYYYYY,2024-06-18 @. 2,XXXXX,YYYYYY,2024-06-18 @.

I found a bug in the way related to the optional custom_key parameter (PR

175 https://github.com/google-marketing-solutions/Tightlock/pull/175 ).

That said, this parameter is only optional if you have not set it up in your Ads account.

All that being said, can you provide additional information about the error you're getting, since I was not able to reproduce? If you can also provide redacted information about the setup of your destination (i.e. if you are providing all the required fields in the UI) that would be great.

I'm still trying to find a good way to debug remotely...one option would be for you to temporarily invite me @.***) to your Ads account, so that I can try sending a test hit and see if I can reproduce the problem.

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2183206450, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVNZCM5ULGOXZ2SZIHI3ZIRTQLAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTGIYDMNBVGA . You are receiving this because you were mentioned.Message ID: @.***>

[caiotomazelli]

Hey Jacob, I can't see the images when they come from email attachments. Can you post them directly in Github? Thanks!

[jbrs360]

My bad.

[caiotomazelli]

Hi Jacob,

Can you send me the full error log you are getting? I'm trying to query the log database for "The required field was not present." with no luck, so it would be good to have more details.

[jbrs360]

How do I get the full error log? The interface does not return any.

[caiotomazelli]

If you can provide a screenshot of the error you're getting in the interface (the "The required field was not present." that you mentioned) that would already be useful.

If you want to have the full log, you can:

1) Go to "Compute Engine" in your cloud project and SSH into the tightlock deployed machine (might require relaxing our default SSH restrictions. You can allow port 22 to 0.0.0.0/0 following the firewall rules update instructions here: https://cloud.google.com/firewall/docs/using-firewalls#updating_firewall_rules)

2) Once you are connected to the instance, you can go to /mnt/disks/tightlock/Tightlock/logs and all execution logs will be in this folder

I understand that these troubleshooting steps are complicated, but for now there is no other way to see them unfortunately. Simplifying this flow is currently in our roadmap though.

[jbrs360]

I had a more technical colleague look into the error log with the guide you sent me. Thanks for the steps! He said it looked like something went wrong with the email column, which made me look further into the data we sent. Some rows were missing an email but had phone numbers and other personal information. Removing the rows with no email made the connection run successfully.

To improve the UI it would be nice to know what went wrong with the imported data. E.g. some of the rows were missing an email although it's not set to be required in the GitHub table.

[caiotomazelli]

Thanks for the update Jacob!

This is definitely a bug. I'll keep this issue open as I investigate.

Thanks again for your patience here and for the help while troubleshooting the issue.

[jbrs360]

No problem Caio!

It would be a nice feature to get a "Successful run"-message with a webhook to e.g. Slack, to follow the different connections and to react as fast as possible if something went wrong.

[jbrs360]

Hi Caio,

I got two questions:

1. Could you make the Run log descending? Having the newest runs in the top would be a better overview.
2. The token I've created keeps expiring: "invalid_grant: Token has been expired or revoked." Any workaround for that?

[caiotomazelli]

Hi Jacob, sorry, missed your questions here.

Answers below:

1) I'll look into having that as an option, thanks for pointing that out. 2) In this link, you can find common reasons for premature token expiration (normally, refresh_token should only expire after 6 months of inactivity). The most common cause of this problem is having your GCP OAuth project with "Testing" publishing status, which causes the token to expire after 7 days. If that's not the case, this could also be related to org specific policies in your GCP project.

[jbrs360]

Thanks, Caio!

We've had it running for 7 days with uploads, no errors. But I cannot see any conversions or value added. By the looks of it the data is not ending up the right place all though I've double checked ids.

On Tue, 30 Jul 2024 at 15:16, Caio Tomazelli @.***> wrote:

Hi Jacob, sorry, missed your questions here.

Answers below:

1. I'll look into having that as an option, thanks for pointing that out.
2. In this link https://developers.google.com/identity/protocols/oauth2#expiration, you can find common reasons for premature token expiration (normally, refresh_token should only expire after 6 months of inactivity). The most common cause of this problem is having your GCP OAuth project with "Testing" publishing status, which causes the token to expire after 7 days. If that's not the case, this could also be related to org specific policies in your GCP project.

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2258328788, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVN3DCZ5FMSVUEVYAJFLZO6G4RAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGMZDQNZYHA . You are receiving this because you were mentioned.Message ID: @.***>

[jbrs360]

Hi Caio,

Just bumping this one in case you missed it. The uploaded data isn't showing in the platform. Could you have a look?

On Tue, 30 Jul 2024 at 15:35, Jacob Brix @.***> wrote:

Thanks, Caio!

We've had it running for 7 days with uploads, no errors. But I cannot see any conversions or value added. By the looks of it the data is not ending up the right place all though I've double checked ids.

On Tue, 30 Jul 2024 at 15:16, Caio Tomazelli @.***> wrote:

Hi Jacob, sorry, missed your questions here.

Answers below:

1. I'll look into having that as an option, thanks for pointing that out.
2. In this link https://developers.google.com/identity/protocols/oauth2#expiration, you can find common reasons for premature token expiration (normally, refresh_token should only expire after 6 months of inactivity). The most common cause of this problem is having your GCP OAuth project with "Testing" publishing status, which causes the token to expire after 7 days. If that's not the case, this could also be related to org specific policies in your GCP project.

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2258328788, or unsubscribe https://github.com/notifications/unsubscribe-auth/BACDVN3DCZ5FMSVUEVYAJFLZO6G4RAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGMZDQNZYHA . You are receiving this because you were mentioned.Message ID: @.***>

[caiotomazelli]

Hey @jbrs360 sorry for the delay.

Unfortunately there are many reasons why you're not seeing the data. Normally, when there are no errors, this is unrelated to Tightlock.

I would advise following the steps in the comment above for checking the logs again and see what is the raw response from the API in the success cases and share here if you still can't find the reason.

Cheers, Caio

[jbrs360]

Hey @caiotomazelli

I've heard back from API support that they were able to check the logs and can see Google Ads API v15 is being used. As v15 was deprecated on February 21, 2024, and will sunset by September 25, 2024, they suggest to upgrade to the latest version 'v17' as soon as possible to benefit from new features.

Can you please try to upgrade to the latest version of Google Ads API?

[caiotomazelli]

Hi Jacob,

We updated the version to v17 about 3 months ago in PR #170. Can you update your deployment to the latest version and try again?

Hope this solves the issue!

[caiotomazelli]

Hi folks,

We identified that an operation was missing in the GAds SSI operation (Thanks to @piariachi). As soon as #184 is merged, this should be fixed. Be sure to update your instance of Tightlock after the merge.

Sorry for the delay in figuring this out, hope this solves the problems on your end!

[jbrs360]

I made a completely new install and set up a connection to the same BQ data. We're receiving the following error: ErrorNameIDMap.ADS_SSI_HOOK_ERROR_MISSING_MANDATORY_FIELDS

BQ schema is still: Note: It is missing the state field. However, it isn't required.

[jbrs360]

Yesterday and today Tightlock did not report any issue with the uploaded data. However, I still don't see any indication or change for the conversion action values in Google Ads.

[piariachi]

Hey @jbrs360 , Could you check the status of the jobs using the below query using the searchStream API endpoint? You could run this in the UI (under Try This Method section): https://developers.google.com/google-ads/api/rest/reference/rest/v17/customers.googleAds/searchStream

SELECT offline_user_data_job.id, offline_user_data_job.external_id, offline_user_data_job.type, offline_user_data_job.failure_reason, offline_user_data_job.resource_name, offline_user_data_job.status FROM offline_user_data_job

[jbrs360]

Hi @piariachi ,

These are the last two jobs (I believe)

    {
      "offlineUserDataJob": {
        "resourceName": "customers/*****/offlineUserDataJobs/*****",
        "type": "STORE_SALES_UPLOAD_FIRST_PARTY",
        "status": "SUCCESS",
        "id": "42264066387"
      }
    },
    {
      "offlineUserDataJob": {
        "resourceName": "customers/*****/offlineUserDataJobs/*****",
        "type": "STORE_SALES_UPLOAD_FIRST_PARTY",
        "status": "RUNNING",
        "id": "42288605575"
      }
    }
  ],
  "fieldMask": "offlineUserDataJob.id,offlineUserDataJob.externalId,offlineUserDataJob.type,offlineUserDataJob.failureReason,offlineUserDataJob.resourceName,offlineUserDataJob.status",
  "requestId": "ysn9wy1lhvLP2hWPstis5A",
  "queryResourceConsumption": "54"
}
]

I got a Collab running looking into the jobs and this is an overview of all jobs:

-----
FAILED: 60
PENDING: 311
SUCCESS 628
RUNNING: 1
-----
TOTAL:  1000
-----

It would be great to match the created job id from Tightlock with the ids from the search stream. Could this also somehow be integrated into Tightlock?

[piariachi]

Hello,

It's great that you got a SUCCESS status, I also expect the RUNNING job to go through soon.

Could you have a look at the best practices part of the below link and make sure your uploads meet the criteria?

https://support.google.com/google-ads/answer/10018944?hl=en

Please also make sure you keep uploading data regularly (weekly or daily).

Pia

On Wed, Oct 9, 2024, 3:26 PM Jacob @.***> wrote:

Hi @piariachi https://github.com/piariachi ,

These are the last two jobs (I believe)

{
  "offlineUserDataJob": {
    "resourceName": "customers/*****/offlineUserDataJobs/*****",
    "type": "STORE_SALES_UPLOAD_FIRST_PARTY",
    "status": "SUCCESS",
    "id": "42264066387"
  }
},
{
  "offlineUserDataJob": {
    "resourceName": "customers/*****/offlineUserDataJobs/*****",
    "type": "STORE_SALES_UPLOAD_FIRST_PARTY",
    "status": "RUNNING",
    "id": "42288605575"
  }
}

], "fieldMask": "offlineUserDataJob.id,offlineUserDataJob.externalId,offlineUserDataJob.type,offlineUserDataJob.failureReason,offlineUserDataJob.resourceName,offlineUserDataJob.status", "requestId": "ysn9wy1lhvLP2hWPstis5A", "queryResourceConsumption": "54" } ]

— Reply to this email directly, view it on GitHub https://github.com/google-marketing-solutions/Tightlock/issues/166#issuecomment-2402346598, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVVTIG3PAUU3HTBI3WSLUPTZ2UVIHAVCNFSM6AAAAABHX55KWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBSGM2DMNJZHA . You are receiving this because you were mentioned.Message ID: @.***>

[jbrs360]

We're uploading between 2500-5000 transactions everyday with the schema mentioned before.