Problem during fulltextsearch:index "Root mapping definition has unsupported parameters:"

[fillg1]

Hi folks I tried to update my existing nextcloud to elasticsearch 8, but I got a problem during reindexing, even after a fulltextsearch:reset

My versions Nextcloud Hub 6 27.1.3 Full text search 27.0.2 Full text search - Elasticsearch Platform 27.0.4 Full text search - Files 27.0.1 Full text search - Files - Tesseract OCR 27.0.0 Elasticsearch 8.8.2

php occ fulltextsearch:test seems to work, I don't see any errors in the output `` .Testing your current setup: Creating mocked content provider. ok Testing mocked provider: get indexable documents. (2 items) ok Loading search platform. (Elasticsearch) ok Testing search platform. ok Locking process ok Removing test. ok Pausing 3 seconds 1 2 3 ok Initializing index mapping. ok Indexing generated documents. ok Pausing 3 seconds 1 2 3 ok Retreiving content from a big index (license). (size: 32386) ok Comparing document with source. ok Searching basic keywords:

• 'test' (result: 1, expected: ["simple"]) ok
• 'document is a simple test' (result: 2, expected: ["simple","license"]) ok
• '"document is a test"' (result: 0, expected: []) ok
• '"document is a simple test"' (result: 1, expected: ["simple"]) ok
• 'document is a simple -test' (result: 1, expected: ["license"]) ok
• 'document is a simple +test' (result: 1, expected: ["simple"]) ok
• '-document is a simple test' (result: 0, expected: []) ok
• 'document is a simple +test +testing' (result: 1, expected: ["simple"]) ok
• 'document is a simple +test -testing' (result: 0, expected: []) ok
• 'document is a +simple -test -testing' (result: 0, expected: []) ok
• '+document is a simple -test -testing' (result: 1, expected: ["license"]) ok
• 'document is a +simple -license +testing' (result: 1, expected: ["simple"]) ok Updating documents access. ok Pausing 3 seconds 1 2 3 ok Searching with group access rights:
• 'license' - [] - (result: 0, expected: []) ok
• 'license' - ["group_1"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_1","Group_2"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_3","Group_2"] - (result: 1, expected: ["license"]) ok
• 'license' - ["group_3"] - (result: 0, expected: []) ok Searching with share rights:
• 'license' - notuser - (result: 0, expected: []) ok
• 'license' - User number_2 - (result: 1, expected: ["license"]) ok
• 'license' - User3 - (result: 1, expected: ["license"]) ok
• 'license' - User@4 - (result: 1, expected: ["license"]) ok Removing test. ok Unlocking process ok ``

But when I try to reindex all my documents with fulltextsearch:index, and the first PDF file is "hit", I'm getting an error from elasticsearch

org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: Root mapping definition has unsupported parameters: [standard : {dynamic=true, properties={owner={type=keyword}, groups={type=keyword}, circles={type=keyword}, metatags={type=keyword}, source={type=keyword}, title={copy_to=combined, analyzer=keyword, term_vector=with_positions_offsets, type=text}, content={copy_to=combined, analyzer=analyzer, term_vector=with_positions_offsets, type=text}, users={type=keyword}, tags={type=keyword}, provider={type=keyword}, subtags={type=keyword}, links={type=keyword}, combined={analyzer=analyzer, term_vector=with_positions_offsets, type=text}, hash={type=keyword}}}] at org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:385) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:361) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:1339) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:472) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:694) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:470) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:593) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:420) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:299) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:550) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1039) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1004) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:232) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1622) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:357) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1619) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1237) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:357) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1216) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983) ~[elasticsearch-8.8.2.jar:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.8.2.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:840) ~[?:?]

Any idea what is going wrong here?

[linnaea]

https://github.com/nextcloud/fulltextsearch_elasticsearch/blob/22830220d6418f551953ce286bebd2a812108b8a/lib/Service/IndexMappingService.php#L241

Root cause is here, mapping type removal that should've been done during ES7.

There are probably some PRs that needs to be reverted, like #237, otherwise fixing the root cause will cause user/group/circle filtering to always return nothing.

Also I find the acceptance of #246 quite baffling.

https://github.com/nextcloud/fulltextsearch_elasticsearch/pull/246/files#diff-9d002463e43412b20501d12daf6ad95fe0c6cdfd8e17cb3627a57e4c66a05d9d

This changed the behaviour during index creation from "translate some errors" to "log a warning and pretend nothing happens". Shouldn't have been accepted.