search

LinkedInAttic / Cubert

Fast and efficient batch computation engine for complex analysis and reporting of massive datasets on Hadoop
http://linkedin.github.io/Cubert/
Apache License 2.0
246 stars 61 forks source link

Error happened! the Demo don't provide some .avro files for loading n-dims input #5

Open ghost opened 9 years ago

ghost commented 9 years ago

Hi mani,

Error happened!

Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Not a data file. at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.exitProgram(DependencyAnalyzer.java:299) at com.linkedin.cubert.analyzer.physical.PhysicalPlanWalker.walk(PhysicalPlanWalker.java:75) at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.rewrite(DependencyAnalyzer.java:93) at com.linkedin.cubert.ScriptExecutor.rewrite(ScriptExecutor.java:343) at com.linkedin.cubert.ScriptExecutor.main(ScriptExecutor.java:529) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

This project don't have some .avro or .txt files when aggregating a cube using n-dims. So I dont know how to create a .avro file and organize the data structure. Can you upload some files as demo, to show the inner data structure in the .avro file?

Thanks a lot!

theseus yang

mparkhe commented 9 years ago

Hi Theseus,

Under release folderhttps://github.com/linkedin/Cubert/tree/master/release we have provided a simple word count example with text input. With the following releases, we will try to add an example with AVRO and possibly one using CUBE. In the meantime, you could use any standard avro file in cubert script (you could also use the sample ones in Apache Avro download).

You can see code snippets and how to use CUBE statement with different options in the documentation—specifically look at these topics: Aggregation: Cube and Grouping Setshttp://linkedin.github.io/Cubert/concepts/grouping_sets.html?highlight=cube and Aggregation Operators:Cubehttp://linkedin.github.io/Cubert/operators/aggregation.html?highlight=cube.

Thanks, mani

From: GemFireR notifications@github.com<mailto:notifications@github.com> Reply-To: linkedin/Cubert reply@reply.github.com<mailto:reply@reply.github.com> Date: Saturday, May 16, 2015 at 10:00 PM To: linkedin/Cubert Cubert@noreply.github.com<mailto:Cubert@noreply.github.com> Subject: [Cubert] Error happened! the Demo don't provide some .avro files for loading n-dims input (#5)

Hi mani,

Error happened!

Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Not a data file. at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.exitProgram(DependencyAnalyzer.java:299) at com.linkedin.cubert.analyzer.physical.PhysicalPlanWalker.walk(PhysicalPlanWalker.java:75) at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.rewrite(DependencyAnalyzer.java:93) at com.linkedin.cubert.ScriptExecutor.rewrite(ScriptExecutor.java:343) at com.linkedin.cubert.ScriptExecutor.main(ScriptExecutor.java:529) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

This project don't have some .avro or .txt files when aggregating a cube using n-dims. So I dont know how to create a .avro file and organize the data structure. Can you upload some files as demo, to show the inner data structure in the .avro file?

Thanks a lot!

theseus yang

— Reply to this email directly or view it on GitHubhttps://github.com/linkedin/Cubert/issues/5.