search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.27k stars 1.23k forks source link

Vague error message when building startree schema if there is a typo in the aggregate columns #5474

Open prm-dan opened 4 years ago

prm-dan commented 4 years ago

The NullPointerException does not say what field has the typo.

java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:121) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:94) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:123) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:156) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:168) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] Caused by: java.lang.NullPointerException at org.apache.pinot.core.startree.v2.builder.BaseSingleTreeBuilder.(BaseSingleTreeBuilder.java:153) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.OffHeapSingleTreeBuilder.(OffHeapSingleTreeBuilder.java:69) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.getSingleTreeBuilder(MultipleTreesBuilder.java:140) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.build(MultipleTreesBuilder.java:118) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.buildStarTreeV2IfNecessary(SegmentIndexCreationDriverImpl.java:277) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:239) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:199) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:102) ~[pinot-batch-ingestion-standalone-0.4.0-SNAPSHOT-shaded.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.run(SegmentGenerationJobRunner.java:190) ~[pinot-batch-ingestion-standalone-0.4.0-SNAPSHOT-shaded.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:119) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] ... 4 more Exception caught: java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:121) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:94) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:123) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:156) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:168) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] Caused by: java.lang.NullPointerException at org.apache.pinot.core.startree.v2.builder.BaseSingleTreeBuilder.(BaseSingleTreeBuilder.java:153) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.OffHeapSingleTreeBuilder.(OffHeapSingleTreeBuilder.java:69) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.getSingleTreeBuilder(MultipleTreesBuilder.java:140) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.startree.v2.builder.MultipleTreesBuilder.build(MultipleTreesBuilder.java:118) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.buildStarTreeV2IfNecessary(SegmentIndexCreationDriverImpl.java:277) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:239) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:199) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:102) ~[pinot-batch-ingestion-standalone-0.4.0-SNAPSHOT-shaded.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.run(SegmentGenerationJobRunner.java:190) ~[pinot-batch-ingestion-standalone-0.4.0-SNAPSHOT-shaded.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:119) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-b40dd992874f9bc38b911870e041a8f6e24c3776] ... 4 more

kishoreg commented 4 years ago

can you paste the sample config

prm-dan commented 4 years ago

I'm working on a diff.

Here's an example of the copy/paste issue (simplified since I don't want to post the full config).

in the table config:

    {
      "tableName": "metrics",
...
      "tableIndexConfig" : {
        "starTreeIndexConfigs": [
          {
...
            "functionColumnPairs": [
              "SUM__impressions",
...
            ]
          },

then in the metrics schema:

    {
      "schemaName": "metrics",
      "metricFieldSpecs": [
...
        {
          "name": "SUM__impressions",
          "dataType": "LONG"
        },
...
      ]
    }
prm-dan commented 4 years ago

https://github.com/apache/incubator-pinot/pull/5476