search

vertica / r-dataconnector

Distributed Data Connector R package
Other
7 stars 3 forks source link

Add globbing for reading orc files #4

Open dselivanov opened 8 years ago

dselivanov commented 8 years ago

It would be nice feature. Will try to investigate into this.

jorgemarsal commented 8 years ago

There's an HDFS globber implemented in https://github.com/vertica/r-dataconnector/blob/00839fe7eba3e8b2fa7007b06189b2e29f3ec3ab/dataconnector/src/distributed-data-connector/ddc/hdfsutils/src/hdfsutils.cpp

You can pass a pattern and the function returns a list of filenames: std::vectorstd::string ddc::hdfsutils::HdfsGlobber::glob(const std::string& pat);

There's also a trivial version for the local filesystem: https://github.com/vertica/r-dataconnector/blob/00839fe7eba3e8b2fa7007b06189b2e29f3ec3ab/dataconnector/src/distributed-data-connector/ddc/base/src/utils.cpp

std::vectorstd::string globpp(const std::string &pat);

Should be easy to add the globbing feature reusing that functionality.