tentacles_api Cloud Function crashes and times out while sending data to Google Ads

[RuslanBergenov]

Background

We use Tentacles to send data to Google Ads API.

We last reinstalled Tentacles on Feb 12, 2024 which is fairly recently.

Issue

Starting on March 15, 2024, tentacles_api Cloud Function started timing out and crashing with the below error.

[T.MAIN] Error in API[ACLC], config[test_NC_pltv_2_2]:  Error: Task[KjTtfB4HIzfCPqPjyLcG] has been processed 4 times.
    at TentaclesTaskOnFirestore.start (/workspace/src/tentacles_task/tentacles_task_firestore.js:182:34)
    at async sendApiData (/workspace/src/tentacles.js:461:30)

Function execution took 539999 ms, finished with status: 'timeout'

The function appears to be restarting and different instances of it run for 40 minutes.

Before these issues started, usually the function would just run for under 5 minutes and do all the work and succeed.

When I run this query:

SELECT
  DATETIME(TIMESTAMP(createdAt), "US/Mountain") createdAtMST,
  *,
  SAFE_DIVIDE(numberOfFailed, numberOfLines) error_percent
FROM
  adswerve-355706.tentacles.TentaclesTask
WHERE
  DATE(TIMESTAMP(createdAt), "US/Mountain") >= "2024-03-01"
ORDER BY
  createdAt DESC

I see that starting from March 15, there are NULLs in numberOfLines and numberOfFailed. Error says: Error: Task[KjTtfB4HIzfCPqPjyLcG] has been processed 4 times.

When examining data in Google Ads UI, it appears that some data is coming thru, but I am almost certain it's not all data, only part of it.

Questions

Is it a known issue? Can you replicate it on your end? Can it be fixed?

Steps to replicate the issue

• Try sending some test data to Google Ads API endpoint.
• Examine Cloud Funtion logs.
• Run the query for TentaclesTask (see above)

[lushu]

Hi,

We have been informed that Google Ads API has longer latency and it caused Tentacles tasks timeout. Tentacles will automatically retry failed tasks for 3 times. However it couldn't recover as the API performance change. That's why there are errors complaining Task has been processed 4 times.

Could you please try to limit the size of Tentacles task? e.g. apply a size in the filename to make the size of each chunk smaller for a task. Please feel free to reach out if you have further questions.

[lushu]

Alternatively, a qps can be added to config like this to control the default chunk size for a task:

{
+  "qps": "0.04",
  "customerId": "[YOUR-GOOGLE-ADS-ACCOUNT-ID]",
  "loginCustomerId": "[YOUR-LOGIN-GOOGLE-ADS-ACCOUNT-ID]",
  "developerToken": "[YOUR-GOOGLE-ADS-DEV-TOKEN]",
  "debug": false,
  "adsConfig": {
    "conversionAction": "[YOUR-CONVERSION-ACTION-NAME]",
    "conversionValue": "[YOUR-CONVERSION-VALUE]",
    "currencyCode": "[YOUR-CURRENCY-CODE]",
    "userIdentifierSource": "[USER_IDENTIFIER_SOURCE]",
    "customVariableTags": "[YOUR-CUSTOM-VARIABLE-TAGS]",
    "consent": {
      "adUserData": "GRANTED|DENIED|undefined",
      "adPersonalization": "GRANTED|DENIED|undefined"
    }
  }
}

[RuslanBergenov]

Thank you! What's the recommended value for the size parameter in the filename for Google Ads?

[lushu]

Thank you! What's the recommended value for the size parameter in the filename for Google Ads?

It depends on the average length of lines in your data file. Based on my latest tests, Tentacles can send out ~20,000 conversions(lines) in one task. So I think you can have a check of your data file and estimate the size of 20,000 lines.

[RuslanBergenov]

Thank you for your prompt help. The issue went away on its own. If it comes back, I will set up the chunk size.