search

mozilla / metric-hub

Central hub for metric definitions that are considered the source of truth
https://mozilla.github.io/metric-hub
Mozilla Public License 2.0
13 stars 13 forks source link

Add clearer error message for case of misnamed analysis periods #354

Open danielkberry opened 8 months ago

danielkberry commented 8 months ago

When a user writes week = [...] instead of weekly = [...], they're met with an incomprehensible metric-config-parser error message (reproduced below). For debugging, it would be great if a cleaner error was thrown in this case.

Exception Group Traceback (most recent call last): | File "/usr/local/bin/jetstream", line 8, in | sys.exit(cli()) | File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call | return self.main(args, kwargs) | File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main | rv = self.invoke(ctx) | File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke | return _process_result(sub_ctx.command.invoke(sub_ctx)) | File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke | return ctx.invoke(self.callback, ctx.params) | File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke | return __callback(args, kwargs) | File "/usr/local/lib/python3.10/site-packages/jetstream/cli.py", line 1160, in validate_config | entity = entity_from_path(config_file, is_private) | File "/usr/local/lib/python3.10/site-packages/metric_config_parser/config.py", line 268, in entity_from_path | spec=AnalysisSpec.from_dict(config_dict), | File "/usr/local/lib/python3.10/site-packages/metric_config_parser/analysis.py", line 48, in from_dict | return converter.structure(d, cls) | File "/usr/local/lib/python3.10/site-packages/cattrs/converters.py", line 332, in structure | return self._structure_func.dispatch(cl)(obj, cl) | File "", line 40, in structure_AnalysisSpec | if errors: raise c_cve('While structuring ' + 'AnalysisSpec', errors, __cl) | cattrs.errors.ClassValidationError: While structuring AnalysisSpec (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "", line 6, in structure_AnalysisSpec | res['metrics'] = c_structure_metrics(o['metrics'], __c_type_metrics) | File "/usr/local/lib/python3.10/site-packages/metric_config_parser/metric.py", line 380, in | converter.register_structure_hook(MetricsSpec, lambda obj, _type: MetricsSpec.from_dict(obj)) | File "/usr/local/lib/python3.10/site-packages/metric_config_parser/metric.py", line 322, in from_dict | params["definitions"] = { | File "/usr/local/lib/python3.10/site-packages/metric_config_parser/metric.py", line 324, in | {"name": k, dict((kk.lower(), vv) for kk, vv in v.items())}, MetricDefinition | AttributeError: 'list' object has no attribute 'items' | Structuring class AnalysisSpec @ attribute metrics +------------------------------------

mikewilli commented 7 months ago

Let's solve this and other similar metric-hub schema issues by validating configs with metric-config-parser in CI.