[feature request] Use pygtail to read the job scheduler's log files

[eburgueno]

Hi David, In your instructions you suggest using logtail for large log files. Couldn't lava-import-openlava make use of something like pygtail instead of opening the log files directly? I don't think pygtail returns a file handler object like open() does, so incorporating it may not be straightforward, but it would definitely be useful. Specially when used in conjunction with --tail_log. Thanks, E.

[irvined1982]

Hi,

I've been considering the same thing, or something similar for a while, definitely a possibility, I'll look into it some more, I've wanted a way to restart imports easily. Working on something else for the next few weeks, but will look into this afterwards.

Thanks

David

On Fri, Nov 14, 2014 at 3:43 PM, eburgueno notifications@github.com wrote:

Hi David, In your instructions https://www.clusterfsck.io/static/lavaflow/import.html#lava-import-openlava you suggest using logtail for large log files. Couldn't lava-import-openlava make use of something like pygtail https://pypi.python.org/pypi/pygtail instead of opening the log files directly? I don't think pygtail returns a file handler object like open() does, so incorporating it may not be straightforward, but it would definitely be useful. Specially when used in conjunction with --tail_log. Thanks, E.

— Reply to this email directly or view it on GitHub https://github.com/irvined1982/lavaflow/issues/13.