search

imperva / incapsula-logs-downloader

A Python script for downloading log files from Incapsula
MIT License
30 stars 35 forks source link

Starting download from more recent file #47

Closed adm-guerinw closed 1 year ago

adm-guerinw commented 1 year ago

Hi All I have logs going back ages, I was pulling them down using a Splunk app but now need to move to python due to python3 incompatibility in my current app. Im trying to start the download using a more recent file that is there using the lastknowndownloadfieldid.txt but I'm running into an error.

[splunk@prdsplkfwd01 script]$ cat LastKnownDownloadedFileId.txt 6697_1031703.log

[splunk@prdsplkfwd01 script]$ python3 LogsDownloader.py -c /opt/incapsula_logs/script -l /opt/incapsula_logs/script/applogs 2022-08-02 16:20:36,441 INFO LogsDownloader initializing is done Exception in thread process_thread: Traceback (most recent call last): File "/usr/lib64/python3.6/threading.py", line 937, in _bootstrap_inner self.run() File "/usr/lib64/python3.6/threading.py", line 885, in run self._target(*self._args, **self._kwargs) File "LogsDownloader.py", line 142, in get_log_files next_file = self.last_known_downloaded_file_id.get_next_file_name() File "LogsDownloader.py", line 445, in get_next_file_name curr_log_file_id = int(curr_log_file_name_arr[1].rstrip(".log")) + 1 + skip_files ValueError: invalid literal for int() with base 10: '1031703.log\n'

is it something in the formatting of the LastKnownDownloadedFileId.txt file?

adm-guerinw commented 1 year ago

fixed this, you just need to drop ".log" from the filename in LastKnownDownloadedFileId.txt