apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.26k stars 1.23k forks source link

Error while trying to run quick-start-offline on windows #4134

Closed pooja2702 closed 5 years ago

pooja2702 commented 5 years ago

While running quick-start-offline.sh in a Windows 10 environment, the following exception occurs: quick-start-offline.sh*

Could not construct uri from path C:\Users\Pooja\AppData\Local\Temp\PinotController Caught internal server exception while uploading segment org.apache.pinot.controller.api.resources.InvalidControllerConfigException: Caught exception while initializing file upload path provider at org.apache.pinot.controller.api.resources.FileUploadPathProvider.(FileUploadPathProvider.java:90) at org.apache.pinot.controller.api.resources.PinotSegmentUploadRestletResource.uploadSegment(PinotSegmentUploadRestletResource.java:269) at org.apache.pinot.controller.api.resources.PinotSegmentUploadRestletResource.uploadSegmentAsMultiPartV2(PinotSegmentUploadRestletResource.java:469) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) at org.glassfish.jersey.server.model.ResourceMethodInvoker.lambda$apply$0(ResourceMethodInvoker.java:381) at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2$1.run(ServerRuntime.java:819) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2.run(ServerRuntime.java:814) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\Users\Pooja\AppData\Local\Temp\PinotController at org.apache.pinot.controller.ControllerConf.getUriFromPath(ControllerConf.java:180) at org.apache.pinot.controller.api.resources.FileUploadPathProvider.(FileUploadPathProvider.java:58) ... 26 more Caused by: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\Users\Pooja\AppData\Local\Temp\PinotController at java.net.URI$Parser.fail(URI.java:2848) at java.net.URI$Parser.checkChars(URI.java:3021) at java.net.URI$Parser.parse(URI.java:3058) at java.net.URI.(URI.java:588) at org.apache.pinot.controller.ControllerConf.getUriFromPath(ControllerConf.java:172) ... 27 more Exception in thread "main" org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 500 (Internal Server Error) with reason: "Caught internal server exception while uploading segment" while sending request: http://172.16.17.239:9000/v2/segments to controller: Pooja-PC, version: 0.2.0-SNAPSHOT-55f68ce0311cc28ee0fa29ef15f44b5526ed5b79 at org.apache.pinot.common.utils.FileUploadDownloadClient.sendRequest(FileUploadDownloadClient.java:324) at org.apache.pinot.common.utils.FileUploadDownloadClient.uploadSegment(FileUploadDownloadClient.java:431) at org.apache.pinot.common.utils.FileUploadDownloadClient.uploadSegment(FileUploadDownloadClient.java:446) at org.apache.pinot.tools.admin.command.UploadSegmentCommand.execute(UploadSegmentCommand.java:130) at org.apache.pinot.tools.admin.command.QuickstartRunner.pushSegment(QuickstartRunner.java:191) at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:169) at org.apache.pinot.tools.Quickstart.main(Quickstart.java:224)

sunny19930321 commented 5 years ago

construct

It looks like this path could not be created。 C:\Users\Pooja\AppData\Local\Temp\PinotController

chenboat commented 5 years ago

The window file system's character "\" is not a valid character when passed directly to the following URI constructor: https://github.com/apache/incubator-pinot/blob/0680437c7619da3da1acd23445c0661d6a113f96/pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java#L174

pooja2702 commented 5 years ago

Yes, I go around it by changing the line to : URI uri = Paths.get(path).toUri();

Works for Windows and should be OS independent.

Neel-rishabhsoft commented 4 years ago

Hello @pooja2702 @chenboat I am facing the same URI issue on windows I can not find this line containing URI in the above mentioned path for current version of pinot Although I found the problem to be in ControllerStarter for me and I think I found the URI to be changed I was getting the following error: Start zookeeper at localhost:2123 in thread main Executing command: StartController -clusterName QuickStartCluster -controllerHost 192.168.42.42 -controllerPort 9000 -dataDir C:\cygwin64\tmp\PinotController -zkAddress localhost:2123 Caught exception while starting controller, exiting. java.lang.RuntimeException: Caught exception while initializing ControllerFilePathProvider at org.apache.pinot.controller.ControllerStarter.initControllerFilePathProvider(ControllerStarter.java:433) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.ControllerStarter.setUpPinotController(ControllerStarter.java:271) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.ControllerStarter.start(ControllerStarter.java:231) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:163) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.tools.admin.command.QuickstartRunner.startControllers(QuickstartRunner.java:97) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:132) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:150) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.tools.Quickstart.main(Quickstart.java:209) [pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] Caused by: org.apache.pinot.controller.api.resources.InvalidControllerConfigException: Caught exception while initializing file upload path provider at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.(ControllerFilePathProvider.java:107) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.init(ControllerFilePathProvider.java:49) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.ControllerStarter.initControllerFilePathProvider(ControllerStarter.java:431) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] ... 7 more Caused by: java.lang.IllegalArgumentException: Illegal URI path: C:\cygwin64\tmp\PinotController at org.apache.pinot.common.utils.URIUtils.getUri(URIUtils.java:47) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.(ControllerFilePathProvider.java:67) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.init(ControllerFilePathProvider.java:49) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.ControllerStarter.initControllerFilePathProvider(ControllerStarter.java:431) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] ... 7 more Caused by: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\cygwin64\tmp\PinotController at java.net.URI$Parser.fail(URI.java:2915) ~[?:?] at java.net.URI$Parser.checkChars(URI.java:3086) ~[?:?] at java.net.URI$Parser.parse(URI.java:3122) ~[?:?] at java.net.URI.(URI.java:600) ~[?:?] at org.apache.pinot.common.utils.URIUtils.getUri(URIUtils.java:40) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.(ControllerFilePathProvider.java:67) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.api.resources.ControllerFilePathProvider.init(ControllerFilePathProvider.java:49) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] at org.apache.pinot.controller.ControllerStarter.initControllerFilePathProvider(ControllerStarter.java:431) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-6f384eaae973978c66f4032e02659d5a5947ebc4] ... 7 more

D:\pinot\incubator-pinot\pinot-controller\src\main\java\org\apache\pinot\controller\api\resources\ControllerFilePathProvider.java So I tried changing following code :

_dataDirURI = URIUtils.getUri(dataDir); to _dataDirURI = URIUtils.get(dataDir).toUri();

But the problem is still persisting. Could you please provide your insights on this?

RogerGMartins commented 3 years ago

Hi can this be patched in the latest version? The problem persists and I don't see any commit doing the fix.

cedricbarbin commented 2 years ago

A working patch for 0.11 running on Windows with GitBash

Class : org.apache.pinot.common.utils.URIUtils Line 41 : String path = "file:/" + getPath(basePath, parts).replace("\", "/");