search

JuliaParallel / Blocks.jl

A framework to represent chunks of entities and parallel methods on them.
MIT License
30 stars 6 forks source link

Is it still alive? #14

Open MaciekLeks opened 9 years ago

MaciekLeks commented 9 years ago

Hello, you've done a great job. How does it correspond to Elly?

tanmaykm commented 9 years ago

Blocks used to work with the older HDFS.jl, but it's not fully ported to Elly.jl yet. I had started doing something with it at https://github.com/tanmaykm/HadoopBlocks.jl, but it's not complete yet.

MaciekLeks commented 9 years ago

Great to hear!... Blocks with HDFS does not work in my case. Have you thouth about somethink like Spark,...Julia Spark?:)

tanmaykm commented 9 years ago

@MaciekLeks I updated Blocks.jl and Elly.jl and HadoopBlocks.jl should work fine now. It uses Elly.jl instead of HDFS.jl. This needs Julia 0.4, and you may need to do Pkg.checkout of the packages for the recent fixes. I will tag new releases after few more checks.

Blocks implementation with HDFS.jl package is still at hdfs_blocks.jl, but is not loaded with Blocks.jl anymore. It may still be working though, and you could try by copying the code over. The intent is to use Elly.jl in future for HDFS, which is pure Julia and does not have any external dependencies.

There have been some discussions on Julia-Spark integration here:

MaciekLeks commented 9 years ago

@tanmaykm thank you for the answer. I could not add "HadoopBlocks", because of the error: "julia> Pkg.add("HadoopBlocks") ERROR (unhandled task failure): unknown package HadoopBlocks ..."

As to the Spark. IMHO, Spark is a recognizable brand , but Julia should not be part of it. Julia is not a Python. Julia is strong enough to be a leader and easily beat up children of JVM. HDFS yes, but "jlSpark" no:)

oschulz commented 7 years ago

@tanmaykm : Is this project still active (regarding Julia 0.6 support, etc.)?

schlichtanders commented 4 years ago

Just found this project as I was looking for finally having a DDataFrame. It seems this once existed here, as part of a more general distribution setting.

Is there a current alternative?

vchuravy commented 4 years ago

You might want to look at JuliaDB.jl