Collection merging is not working.

[apuzyrevsky]

Describe the bug Collection merging is not working. It returns one of collection.

To Reproduce Steps to reproduce the behavior:

1. Add two backends that return collections without mapping or grouping.
2. Trigger endpoint.

Expected behavior Returning two merged collections.

Actual behavior Returns one of collection.

Additional context Grouping or mapping working although.

[apuzyrevsky]

My further investigation of the krakend app got me an idea that actually the issue in the same naming. Two collections both returning "collection" object and then service do not merge it. For example, if I map two different endpoints to the same name, it also wouldn't merge these endpoints.

[kpacha]

this is not a bug. merging responses with shared property names will result in the slowest overwriting the fastest one. you can change the property name of one of the backends with https://www.krakend.io/docs/backends/data-manipulation/#mapping

[apuzyrevsky]

this is not a bug. merging responses with shared property names will result in the slowest overwriting the fastest one. you can change the property name of one of the backends with https://www.krakend.io/docs/backends/data-manipulation/#mapping

I got it, thanks. Is it possible to merge two backends to one property name?

[kpacha]

yes, with the brand new append operation from the flatmap component. it's not yet documented but you can see here how it works: https://github.com/devopsfaith/krakend-documentation/issues/77

[apuzyrevsky]

On what particular level should I apply it? on the on of the backends or on the endpoint?

[kpacha]

as the documentation of the component says:

The flatmap component is part of the krakend proxy operation, so it needs to be included as an extra_config inside the backend configuration. The namespace is github.com/devopsfaith/krakend/proxy.

https://www.krakend.io/docs/backends/flatmap/#flatmap-configuration

[apuzyrevsky]

  "endpoints": [
    {
      "endpoint": "/clocktower/",
      "method": "GET",
      "output_encoding": "json",
      "extra_config": {},
      "backend": [
        {
          "url_pattern": "/summary/",
          "host": ["http://os.api.com"],
          "encoding": "json",
          "method": "GET",
          "extra_config": {},
          "is_collection": true,
          "mapping": {
            "collection": "instances"
          }
        },
        {
          "url_pattern": "/summary/",
          "host": ["http://ps.api.com"],
          "encoding": "json",
          "method": "GET",
          "mapping": {
            "collection": "ps_instances"
          },
          "extra_config": {
            "github.com/devopsfaith/krakend/proxy": {
              "flatmap_filter": [
                {
                  "type": "append",
                  "args": ["ps_instances", "instances"]
                }
              ]
            }
          },
          "is_collection": true
        }
      ]
    }

With the following config I receiving from krakend two collections ("collection" and "instances") what am I doing wrong?

[apuzyrevsky]

As I understood I can't work with a mapped name of collection within flatmap_filter, right? So how can I merge two same-named collections?

[apuzyrevsky]

Sorry for disrupting, but my question still actual, could someone please help me?

[kpacha]

my bad, so far it is not possible to manipulate the response after the partials are merged

you'll need to use 2 endpoints: one merging the partials responses and another manipulating the merged response from the previous endpoint (flatmap operation)

[github-actions[bot]]

This issue was marked as resolved a long time ago and now has been automatically locked as there has not been any recent activity after it. You can still open a new issue and reference this link.