Does PigPen support loading file in customized format?

[ljie-PI]

Hello team,

I am trying to use PigPen in my work. But I have some troubles when I load input data. Sometimes my input data is not a text file. It may be sequence files or files in other customized format. Is there any solution for this?

[mbossenbroek]

Yes there is - basically, we wrap the other storage as another pigpen command.

Here's a post from the pigpen-support google group that outlines how to do this: https://groups.google.com/forum/#!msg/pigpen-support/_yxTmuuqaUo/Ia_9Xu-InGcJ

And here's the code sample on gist (gmail dropped the formatting):

https://gist.github.com/mbossenbroek/8461143

Let me know if the example isn't clear or if you have any other questions.

Thanks, Matt

On Tuesday, January 21, 2014 at 8:21 AM, Rocky Liu wrote:

Hello team, I am trying to use PigPen in my work. But I have some troubles when I load input data. Sometimes my input data is not a text file. It may be sequence files or files in other customized format. Is there any solution for this?

— Reply to this email directly or view it on GitHub (https://github.com/Netflix/PigPen/issues/5).

[ljie-PI]

Thank you, Matt. I think that's what I need.