search

RevolutionAnalytics / rhdfs

A package that allows R developers to use Hadoop HDFS
64 stars 73 forks source link

HDP2.0 / Hadoop 2 compatability #5

Open chrili opened 11 years ago

chrili commented 11 years ago

From the wiki:

model <- 3 modelfilename <- "my_smart_unique_name" modelfile <- hdfs.file(modelfilename, "w") hdfs.write(model, modelfile) [1] TRUE hdfs.close(modelfile) Error in fh$sync : no field, method or inner class called 'sync'

Is 2.0 support in the works?

piccolbo commented 11 years ago

How did you establish the link between this error and what you call "Hadoop 2 compatability"? What versions of rhdfs and hadoop were you running?

chrili commented 11 years ago

rhdfs 1.0.7

Subversion https://github.com/apache/hadoop-common.git -r b906b9931c14c6fb67eb921ac6667f894820bc1a Compiled by chrili on 2013-10-23T00:19Z Compiled with protoc 2.5.0 From source with checksum 986d2a389e73c5509e041bb6d105cd6 This command was run using /opt/apache/share/hadoop/common/hadoop-common-3.0.0-SNAPSHOT.jar

Built from hadoop's head

RevolutionAnalytics commented 11 years ago

As far as I know, Hadoop Common 3.0 hasn't been released yet. We can try something experimental, if you are willing to test it

chrili commented 11 years ago

I'd be very willing to do that!

Hortonworks also recently released their HDP2.0 sandbox: http://hortonworks.com/products/hortonworks-sandbox/

RevolutionAnalytics commented 11 years ago

Please send me a email, and I will provide you with an archive you can use for testing david.champagne@revolutionanalytics.com