Open zhangpengshan opened 4 years ago
Take one example condition say a > 200, but real data in file is 50.0, 30.3, there could be an exeception like ClassCastException from long to double but it is very difficult to identify if not check job logs. We'd better to have such exception and filter stats in console for better debugging.
Segment expansion filter generation is supported:
https://github.com/shifuml/shifu/wiki/Segment-Expansion-for-New-Feature-Generation
While if all records are filtered in one of segment condition, it won't be showed and no such column configurations in CC.json.
It is better to have stats on such filter condition in console outputs or MR job counters.