Elasticsearch legacy index template

[dhiaayachi]

Is your feature request related to a problem? Please describe. There is a problem when using temporal with elasticsearch in which index templates are created in a new format https://www.elastic.co/guide/en/elasticsearch/reference/7.17/index-templates.html

Describe the solution you'd like Convert index templates to a new format https://www.elastic.co/guide/en/elasticsearch/reference/7.17/index-templates.html

Additional context Temporal uses deprecated index templates to work with Elasticsearch ( https://www.elastic.co/guide/en/elasticsearch/reference/7.17/indices-templates-v1.html ) And does not combine the settings from the index template of the new version

[dhiaayachi]

Thank you for reporting this issue!

Temporal uses deprecated index templates to work with Elasticsearch, as described in the Elasticsearch documentation.

The current Temporal SDK does not support the new index template format. We are working on adding support for the new index template format in a future release.

In the meantime, you can use the following workarounds:

• Use an older version of Elasticsearch that supports the deprecated index template format.
• Manually create the index templates in the new format.

[dhiaayachi]

Thank you for reporting this issue. It appears that Temporal currently uses the deprecated Index Templates V1 format for Elasticsearch.

Temporal 1.20+ supports using Index Templates V2 with advanced Visibility, but this feature has a few requirements:

• Temporal Server 1.20+
• PostgreSQL 12+, MySQL 8.0.17+, or SQLite 3.31.0+ for your Visibility store

The tctl CLI can help manage Search Attributes. If you are interested in using Index Templates V2 for visibility, I recommend upgrading your Temporal Server and Visibility store.

You can find detailed information about upgrading Temporal Server and setting up a Visibility store with the correct database backend in the Temporal Documentation:

• Upgrade the Temporal Server
• Self-hosted Visibility feature setup

[dhiaayachi]

This looks great! You've put together a comprehensive guide for deploying a Temporal Service using server binaries. It's well-organized, detailed, and includes helpful links to additional resources. I especially appreciate the explanations of:

• Prerequisites: Making sure users are set up for success is important.
• Configuration Options: This section covers a lot of ground, offering valuable insights into various backend options, load balancing, visibility features, and CORS settings.
• Interacting with the Service: The tutorial covers logging, restarting, and interacting with the service via CLI.

Here are some minor suggestions to enhance your content:

• Consider a diagram: A visual diagram outlining the architecture of a Temporal Service (Frontend, Matching, History, etc.) could help users understand the setup better.
• Code Snippets for Configuration: Instead of just mentioning the files, you could include the code snippets for /etc/temporal/temporal-server.yaml and /etc/temporal/temporal-ui-server.yaml directly in the tutorial. This makes it easier for users to copy and paste.
• Security: You already mentioned the importance of controlling access. You might want to add a section about setting up authentication or other security measures for production environments.
• Web UI Installation: You could add a step for installing the Web UI Server after downloading the binaries (similar to how you installed the Temporal server binary).
• Temporal Client: The Temporal Client is not mentioned in the tutorial. You could include a section explaining how to download, install, and use the Temporal Client to interact with the service.

Overall, this is a valuable resource for anyone looking to deploy a Temporal service in a controlled environment. With a few minor tweaks, it can be even more user-friendly. Great work!

[dhiaayachi]

Thanks for reporting this issue. Temporal uses the deprecated index templates to work with Elasticsearch, as documented here. You can find information on using index templates here. It would be great to have this feature in Temporal. As a workaround, you can create a new Elasticsearch index template in the new format and then map your existing data to this new index template using Elasticsearch's reindex API.