Double check how the schema is updated, log levels fine tune

[tloubrieu-jpl]

💡 Description

When loading data with harvest, the logs are filled with messages like:

[INFO] This LDD already loaded.
[INFO] Updating Elasticsearch schema.
[INFO] Updated 32 fields

I am wondering if we could:

• lower the level of the 2 first messages to DEBUG
• then, does the 3rd message mean we actually update the schema ? I would think if the LDD has already been loaded and is up to date, the schema does not need update.

⚔️ Parent Epic / Related Tickets

No response

[al-niessner]

@tloubrieu-jpl

• [INFO] This LDD already loaded. means that the all of the fields in the LDD are already in -dd. It does not mean that specific LDD is already loaded despite what it says.
• [INFO] Updating Elasticsearch schema. means that the LDD is being checked against the registry mapping -> aka Elasticsearch schema. This is a good example of name evolution.
• '[INFO] Updated 32 fields` means that 32 of the fields in the LDD, while in the -dd, are not in the mapping and they have been added.

Yeah, none of it is very clear as written and way out of date. Not sure it ever can be clear or timely because elastic/open changes names often as do the concepts that support it. All three give you informational status about the harvest process. They really are not for debugging. Well, I certainly did not add them for debugging purposes nor use them for it either.

Given the new information, let me know what you want done.

[tloubrieu-jpl]

Hi @al-niessner, maybe we can be more explicit:

• This LDD already loaded --> -registry-dd index is up to date with LDD
• Updating Elasticsearch schema. --> Updating OpenSearch mapping for index -registry
• Updated 32 fields --> Updated 32 fields in OpenSearch mapping for index -registry

If none of these messages are for debugging, I don't believe the first 2 are useful for the user either, so we could remove them altogether. How does that sound ?