search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.27k stars 1.23k forks source link

Indicate what fields are getting ignored when adding configs #8318

Closed npawar closed 2 years ago

npawar commented 2 years ago

Often users add schema or table config, and have gotten the position of a particular config wrong in the json. For example, transformConfigs put as top level field instead of inside ingestionConfig, or enableDynamicStarTree inside startree configs instead of top level. In such cases, Pinot simply ignores the unknown json fields that are not in their expected positions. This creates much confusion, as users are expecting the config to have some effect, but instead they see no effect. For example, putting transformConfigs as top level, will result in no transformations happening, making one think that the function is not working or there's some bug.

If we could find the delta of what has been saved and what was in the request config and display it to the user, that would help. This should definitely be part of the logs, but also in the response returned from the POST/PUT apis and validate apis

navina commented 2 years ago

I will pick this one up @npawar !

saurabhd336 commented 2 years ago

These PRs add this functionality https://github.com/apache/pinot/pull/8514 https://github.com/apache/pinot/pull/8606