search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.53k stars 1.3k forks source link

Quickstart from console throws ClassNotFoundException for PinotS3 #7211

Open apucher opened 3 years ago

apucher commented 3 years ago

Pinot (Auth)Quickstart throws a ClassNotFoundException for S3PinotFS - and continues regardless. Should S3PinotFS be disabled in quickstart by default?

alex@alex incubator-pinot % java -cp pinot-distribution/target/pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone/target/pinot-batch-ingestion-standalone-0.8.0-SNAPSHOT.jar org.apache.pinot.tools.AuthQuickstart WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. Starting Zookeeper, controller, broker and server 2021/07/26 19:23:59.582 ERROR [PinotFSFactory] [main] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.S3PinotFS with scheme s3 java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?] at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?] at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] 2021/07/26 19:23:59.593 ERROR [QuickstartRunner] [main] Unable to init PinotFS for scheme: s3, class name: org.apache.pinot.plugin.filesystem.S3PinotFS, configs: {region=us-west-2} java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:56) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] Caused by: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?] at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?] at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc] ... 5 more 2021/07/26 19:23:59.598 INFO [StartZookeeperCommand] [main] Executing command: StartZookeeper -zkPort 2123 -dataDir /var/folders/tf/zrrx94ld2n76z80y7w53dmr00000gp/T/1627352639507/baseballStats/rawdata/PinotZkDir 2021/07/26 19:24:00.824 INFO [StartZookeeperCommand] [main] Start zookeeper at localhost:2123 in thread main

xiangfu0 commented 3 years ago

Hmm, we were putting Pinot-s3 as a dependency mostly for a S3 QuickStart. Can you check if the Pinot-s3 plugin is loaded?