Fields and projection API are very inconvenient [DATAMONGO-2637]

[spring-projects-issues]

Zsombor opened DATAMONGO-2637 and commented

We use a couple of projections in our app to limit the amount of data to transfer, with something like this:

aggregationSteps += Aggregation.project(
   "aaaa",
   "bbbb",
   "cccc",
   "dddd"
 ....  (20 more similar lines)

However, in a refactor, we moved a couple of fields into a sub-entity, and blindly adjusted the projection to this:

aggregationSteps += Aggregation.project(
     "aaaa",
     "bbbb",
     "xxx.cccc",
     "dddd"
  ... (20 more similar lines)

Which surprisingly doesn't work as expected.

The smallest solution, which I could find is:

val fields = listOf(
     "aaaa",
     "bbbb",
     "xxx.cccc",
     "dddd"
  ... (20 more similar lines)
).map { Fields.field(it, it) }

aggregationSteps += Aggregation.project(Fields.from(*fields.toTypedArray()))

So the issue is:

• Aggregation.project( String... names)and Fields.fields( String... names) has a surprising default behavior
• There is no API which takes an array of strings, or a list of strings as field names for projections
• There is no API which takes a list of Fields for projections

 

 

---

Affects: 3.0.4 (Neumann SR4)

[divyajnu08]

Is this issue already fixed ? I tried to replicate it but its working fine . Even I checked Aggregation has methods for taking array of strings and list of fields.

[GabeMedrash]

Just want to amplify the first point detailed by OP, with an added use case and additional details.

OP writes:

Aggregation.project(String... names) and Fields.fields(String... names) has a surprising default behavior

Projections can work one of two ways, inclusion or exclusion. In order to support projecting one way or the other with the existing Aggregation API (v3.2.4), I made use of org.springframework.data.mongodb.core.aggregation.ProjectionOperation which differs from OP's approach (at least at a high level, though it delegates to the code paths hit by OP's solution as well). E.g.:

var projectionOperation = new ProjectionOperation()
    .andExclude("aaa.bbb")
    .andExclude("ccc.ddd.eee");

// Or...
var projectionOperation = new ProjectionOperation()
    .andInclude("aaa.bbb")
    .andInclude("ccc.ddd.eee");

As with the solution OP details, however, this does not work with nested paths! The first path segment in all dot notation paths are removed. E.g., the projection { "aaa.bbb": 0} is inexplicably transformed to { "bbb": 0}. This is what I believe OP was referencing related to "surprising default behavior." If not, it certainly surprises me!

I tracked down what I believe is the culprit. I couldn't find a code or commit comment that explains this behavior.

[divyajnu08]

Hi Mark/Christoph

Please find the PR for this issue: https://github.com/spring-projects/spring-data-mongodb/pull/3840

Thanks