elastic / elasticsearch-net

This strongly-typed, client library enables working with Elasticsearch. It is the official client maintained and supported by Elastic.
https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/index.html
Apache License 2.0
11 stars 1.15k forks source link

Attribute mapping 8.x #7168

Closed KoalaBear84 closed 1 year ago

KoalaBear84 commented 1 year ago

Is your feature request related to a problem? Please describe. I'm missing easy possibilities to map certain fields to [Nested] or [Object].

Currently I'm getting the following error:

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Limit of total fields [1000] has been exceeded"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Limit of total fields [1000] has been exceeded"
    },
    "status": 400
}

The issue is that these fields are dynamic, they are the fields of an entity, so I do not control the amount of fields.

Describe the solution you'd like Any way to indicate that certain POCO properties are nested, and should not count as real fields?

Describe alternatives you've considered Manual mapping? It will be not that easy.

Additional context No additional context

Compared to the 7.x client there is really not much documentation.

KoalaBear84 commented 1 year ago

I just tried manual mapping, but this still didn't work and I had to use the following 'hack' to allow 2000 fields. I thought it would exclude the nested ones.

CreateIndexResponse createIndexResponse = elasticsearchClient.Indices.Create(indexName, c => c
    .Settings(s => s
        .Mapping(c => c
            .TotalFields(f => f
                .Limit(2000)
            )
        )
    )
    .Mappings(m => m
        .Properties<AuditLoggingModel>(p => p
            .Nested(nameof(AuditLoggingModel.Entity))
            .Nested(nameof(AuditLoggingModel.EntityOld))
            .Nested(nameof(AuditLoggingModel.Parameters))
        )
    )
);
stevejgordon commented 1 year ago

@KoalaBear84 Apologies, I missed this when triaging the issues previously. I'm not an expert, but as I read it in our docs, nested fields still count towards the total. You might want flattened fields. Attribute mappings are a feature we're dropping in v8, so manual mapping is the correct approach to configure mappings for types.

You need to ensure the mapping property names match the casing used for fields, which by default uses camel casing. It's best to let our inference handle the names with code such as...

.Mappings(m => m
    .Properties<AuditLoggingModel>(p => p
        .Nested(n => n.Entity)
        .Nested(n => n.EntityOld)
        .Nested(n => n.Parameters)
    )
)
KoalaBear84 commented 1 year ago

OK, is this covered anywhere in the documentation right now? Would be nice if it is mentioned anywhere that this isn't supported anymore.

Great, if needed I'll try to use the flattened fields for those dynamic (and too much fields), if they don't need to be indexed.

Thanks, indeed, I will update the code to make sure it will use the inference names.

stevejgordon commented 1 year ago

The docs need a lot of work to cover the new client in-depth. We do call this out in the release notes for now.