Issue with Recovery logic on next_file

[mcvic1rj]

Per the Log download documentation

Send an HTTPS call for each file listed in the index file that you want to download. As new log files are generated, they are numbered sequentially, but may occasionally skip integers.

The logic in the script will retry with the last downloaded +2 (via skip_files), but there are times when the logs are skipping multiple integers, which causes this script to go into an endless loop.

An example excerpt from our index file:

***_***8988.log
***_***8989.log
***_***8992.log
***_***8995.log
***_***8996.log
***_***8999.log
***_***9000.log
***_***9003.log
***_***9005.log
***_***9006.log
***_***9007.log
***_***9008.log

Suggested possible fix, add another catch that will determine if we are taking too long and look at the array position of the last downloaded file and move 1 position forward

[joeymoore]

@mcvic1rj please let me know if this is still and issue. I would recommend upgrading to the new 3.0.0 release that is available and resolves a lot of known issues. Please review the readme if planning to upgrade. https://github.com/imperva/incapsula-logs-downloader/tree/release-3.0.0-beta