Azure / api-management-policy-snippets

Re-usable examples of Azure API Management policies
MIT License
341 stars 155 forks source link

cannot match caching key in cache-lookup-value policy #43

Closed jonathandbyrd closed 4 years ago

jonathandbyrd commented 4 years ago

im doing a version of the simple example in the docs for custom caching using cache-store-value and cache-lookup-value.

i set the key in the store-value section and the key has a pre-pended '2_[whatever ive used for the key]'

so it can never find the value ive stored.

vladvino commented 4 years ago

@maksimkim please comment.

jonathandbyrd commented 4 years ago

Update: Good News! (for me)

this was failing with a cache miss

[policy xml removed for brevity]

<cache-lookup-value key="@("itemnumber" + (string)context.Variables["itemNumber"])" variable-name="itemNumberJson" caching-type="internal" />

<cache-store-value caching-type="internal" key="@("itemnumber" + (string)context.Variables["itemNumber"])"

this now works

<cache-lookup-value key="@((string)context.Variables["itemNumber"])" variable-name="itemNumberJson" caching-type="internal" />

<cache-store-value caching-type="internal" key="@((string)context.Variables["itemNumber"])" />

something about the concatenation it didn't like. It wasn't failing on the "cache-store-value" section, it was storing it, just on the "cache-lookup-value" section, never found it until i removed the prepended constant

harre096 commented 3 years ago

I am having the same issue with a 2_ prefix being added. However I'm not doing string concatenation... Did we ever figure out root cause here?

gabrielbustamante commented 3 years ago

@harre096 @vladvino @jonathandbyrd i have the same problem, always that i use this "cache-lookup-value", for some reason the key has a pre-pended "2_", someone with the same problem?

if i check redis from my backend service (.net) i can matching the values, but if i call redis directly from my Azure gateway, this prefix "2_" always appear

harre096 commented 3 years ago

@gabrielbustamante @jonathandbyrd

Good news from my side! I figured out that my issue was not the "2" prefix at all. Apparently that is just how APIM works behind the scenes: they always prepend the "2'. This is fine I guess, but their logs/docs should really account for this more clearly. In debugging my issue, this seeming issue was a very time-consuming false-positive.

My actual issue was redis firewall rules. I had added a firewall rule to allow my PC which inadvertently ended up blocking APIM. The apim trace logs let me down again by saying "Value stored in cache" when indeed it had been blocked by a firewall rule. {"source":"cache-store-value","timestamp":"2021-10-01T21:03:56.4601530Z","elapsed":"00:00:00.2815404","data":{"message":"Value stored in cache.","key":"2__my_key_name","value":"my_json_paylod"","timeout":"06:00:00"}}

TLDR: Don't trust apim trace. It's useful, but certainly not definitive.

thepolyglotter commented 2 years ago

In my case, cache-store-value is not working, so there is always cache miss. What could be the reason?

Policy:

<set-variable name="partnerInfo" value="@{
            var partnerAccountNumber = context.Variables["partnerAccountNumber"];
            var partnerRegion = context.Variables["partnerRegion"];
            var partnerType = context.Variables["partnerType"];

            return $"{partnerAccountNumber}:{partnerRegion}:{partnerType}";
        }"/>

<cache-store-value caching-type="internal" key="@((string)context.Variables["id"])" value="@((string)context.Variables["partnerInfo"])" duration="10000"/>

Trace:

{
    "source": "cache-lookup-value",
    "timestamp": "2022-03-10T21:58:24.6068604Z",
    "elapsed": "00:00:00.0005265",
    "data": "No appropriate cache found for provided policy configuration. Policy execution will be skipped."
},
{
    "source": "cache-lookup-value",
    "timestamp": "2022-03-10T21:58:24.6068650Z",
    "elapsed": "00:00:00.0005312",
    "data": {
        "message": "Cache lookup resulted in a miss, variable will not be set.",
        "key": "2_7c486765553047aca4f1088c0e5b1b8e",
        "variableName": "partnerInfo"
        }
}
/
/
/
{
    "source": "set-variable",
    "timestamp": "2022-03-10T21:58:25.2051048Z",
    "elapsed": "00:00:00.5987711",
    "data": {
        "message": "Context variable was successfully set.",
        "name": "partnerInfo",
        "value": "674756756:EMEA:CUSTOMER"
        }
},
{
    "source": "cache-store-value",
    "timestamp": "2022-03-10T21:58:25.2051357Z",
    "elapsed": "00:00:00.5988023",
    "data": "No appropriate cache found for provided policy configuration. Policy execution will be skipped."
}

P.S.: My instance is on Developer tier

jvecc commented 2 years ago

In my case, cache-store-value is not working, so there is always cache miss. What could be the reason?

Policy:

<set-variable name="partnerInfo" value="@{
          var partnerAccountNumber = context.Variables["partnerAccountNumber"];
          var partnerRegion = context.Variables["partnerRegion"];
          var partnerType = context.Variables["partnerType"];

          return $"{partnerAccountNumber}:{partnerRegion}:{partnerType}";
      }"/>

<cache-store-value caching-type="internal" key="@((string)context.Variables["id"])" value="@((string)context.Variables["partnerInfo"])" duration="10000"/>

Trace:

{
  "source": "cache-lookup-value",
  "timestamp": "2022-03-10T21:58:24.6068604Z",
  "elapsed": "00:00:00.0005265",
  "data": "No appropriate cache found for provided policy configuration. Policy execution will be skipped."
},
{
  "source": "cache-lookup-value",
  "timestamp": "2022-03-10T21:58:24.6068650Z",
  "elapsed": "00:00:00.0005312",
  "data": {
      "message": "Cache lookup resulted in a miss, variable will not be set.",
      "key": "2_7c486765553047aca4f1088c0e5b1b8e",
      "variableName": "partnerInfo"
      }
}
/
/
/
{
  "source": "set-variable",
  "timestamp": "2022-03-10T21:58:25.2051048Z",
  "elapsed": "00:00:00.5987711",
  "data": {
      "message": "Context variable was successfully set.",
      "name": "partnerInfo",
      "value": "674756756:EMEA:CUSTOMER"
      }
},
{
  "source": "cache-store-value",
  "timestamp": "2022-03-10T21:58:25.2051357Z",
  "elapsed": "00:00:00.5988023",
  "data": "No appropriate cache found for provided policy configuration. Policy execution will be skipped."
}

P.S.: My instance is on Developer tier

I am facing the same problem. I use an existing redis cache and I'd like to simply get the value of one entry. I have not found a way to avoid this prefix. It would be nice to be able to reuse an existing Redis Cache data.

I believe this only works with an internal cache. When using Redis, the actual result in there shows this "2_" prefix.

UmairSyed commented 1 year ago

I am facing the same issue when i am using cache lookup or cache store using internal cache. any resolutions yet?