[BUG] cancellation_time_millis is not getting added in .tasks index

Describe the bug

When a task is cancelled, the .tasks index fails to update because this index has strict dynamic mapping enabled and cancellation_time_millis field is not present in the mappings. The exception logged is below:

[2024-09-13T20:33:48,799][WARN ][o.o.t.TaskManager        ] [8a67d3f648f0be8167ed2246724ec0d2] couldn't store response org.opensearch.replication.task.CrossClusterReplicationTask$CrossClusterReplicationTaskResponse@74ef241c
 StrictDynamicMappingException[mapping set to strict, dynamic introduction of [cancellation_time_millis] within [task] is not allowed]
         at org.opensearch.index.mapper.DocumentParser.parseDynamicValue(DocumentParser.java:835)
         at org.opensearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:684)
         at org.opensearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:457)
         at org.opensearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:415)
         at org.opensearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:520)
         at org.opensearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:541)
         at org.opensearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:443)
         at org.opensearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:415)
         at org.opensearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:137)
         at org.opensearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:92)
         at org.opensearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:256)
         at org.opensearch.index.shard.IndexShard.prepareIndex(IndexShard.java:1188)
         at org.opensearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:1145)
         at org.opensearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:1062)
         at org.opensearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:625)
         at org.opensearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:471)
         at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
         at org.opensearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:535)
         at org.opensearch.action.bulk.TransportShardBulkAction.dispatchedShardOperationOnPrimary(TransportShardBulkAction.java:416)
         at org.opensearch.action.bulk.TransportShardBulkAction.dispatchedShardOperationOnPrimary(TransportShardBulkAction.java:125)
         at org.opensearch.action.support.replication.TransportWriteAction$1.doRun(TransportWriteAction.java:275)
         at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:922)
         at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
         at java.lang.Thread.run(Thread.java:840)

Related component

Other

To Reproduce

Create a cluster with CCR plugin
Create an auto follow rule
delete the auto follow rule
Observe that the exception about .tasks index should appear in logs

Expected behavior

.tasks index should get updated correctly. I see that the .tasks index is created using the mappings defined here: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/resources/org/opensearch/tasks/task-index-mapping.json

These mappings should contain cancellation_time_millis fields as well.

Additional Details

Plugins Please list all plugins currently enabled. cross-cluster-replication

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context Add any other context about the problem here.

opensearch-project / OpenSearch