This loader lets you specify a pattern instead of a specific path for files, as well as a start/end date, and it will replace '%DATE%' in the pattern with each date in the range.
Example:
my_logs = LOAD '/path/to/weblogs/%DATE%'
USING com.mozilla.pig.load.SnippetDateLoader('2012-01-01','2012-01-31') AS (...);
This will load all the log files for January 2012:
/path/to/weblogs/2012-01-01
/path/to/weblogs/2012-01-02
...
/path/to/weblogs/2012-01-31
This loader lets you specify a pattern instead of a specific path for files, as well as a start/end date, and it will replace '%DATE%' in the pattern with each date in the range.
Example: my_logs = LOAD '/path/to/weblogs/%DATE%' USING com.mozilla.pig.load.SnippetDateLoader('2012-01-01','2012-01-31') AS (...);
This will load all the log files for January 2012: /path/to/weblogs/2012-01-01 /path/to/weblogs/2012-01-02 ... /path/to/weblogs/2012-01-31