Fail / Warn on a topic configured in topic2table.map not present in topics or topics.regex

[Dionakra]

Hi Snowflake Team,

I propose a small change in the validation of the snowflake.topic2table.map property regarding the topics being configured.

We have faced more than once a failed copy/paste between the topics and the snowflake.topic2table.map properties, messing up with casing or some simple miss-typing, generating silent errors in the sinks, as the topics would still be ingested in Snowflake, but not in the desired table, causing some headaches.

So, the change I propose is the following: In the Utils.parseTopicToTable method, validate that, each topic present in the map, matches with, at least, one of the defined topics / regexes, and if a topic is not present, throw a new error informing about the mismatch between the topics being configured as source, and the mappings available, so the developer:

• Makes sure that every topic configured in the snowflake.topic2table.map property is being ingested by the connector.
• Knows if a mismatch happens between the configured topics and the mapping between topics and tables.

[sfc-gh-gjachimko]

thank you for your suggestion. we will add internal ticket to track that.

[sfc-gh-gjachimko]

@Dionakra we've added this to our backlog for Q3. I'll close this issue when we get that delivered.