search

fingltd / 4mc

4mc - splittable lz4 and zstd in hadoop/spark/flink
Other
108 stars 36 forks source link

AVRO support #1

Closed akhil29sep closed 9 years ago

akhil29sep commented 9 years ago

Hi Couple of doubts

  1. How to convert text file to 4mc file format in linux so that it can be used as input to my mapreduce program.
  2. Can 4mc commpression used on AVRO file format..?

Thanks

Kind Regards Akhil Dogra

carlomedas commented 9 years ago
  1. once you build 4mc it provides the command line '4mc' tool that can compress/uncompress ; if you have doubts or issues building, let me know
  2. 4mc is a container, so it can contain anything; if you want to use it with hadoop leveraging scalability, an adapter must be created; e.g. if ElephantBird supports it, I can help you create the bridge between the two.
akhil29sep commented 9 years ago

Thanks : ) , I will try doing what you suggested and will post if i stucks somewhere.

Akhil

carlomedas commented 9 years ago

Most welcome, let me know.