Should locally executed load functions support compression?

[aconbere]

I just lost some time hunting down why my locally executed load of "input.tsv.gz"

(load-tsv "input.tsv.gz")

was returning garbage values. Now of course I realize my error and that when run locally load-tsv hasn't been instrumented to support the compression unlike when run in the pig environment.

This feels like a pretty elementary mistake and one that I wont make again, but it also feels like something that pigpen could have either supported or warned me about ("hey you have a .gz file extension on here and this isn't actually pig!")

[mbossenbroek]

Great point! The netflix extensions we use to look for gz & use the appropriate reader, but I never ported that back to the common loaders. I'll get that fixed soon.

[aconbere]

Rad :)