Update to tv_grab_nl (to reduce cpu-load and more)

[fransg]

Hi Hika, Here's my second contribution. I tested this on my server and my sister's and it works well on both, i.e. same or better performance, whilst reducing cpu-load quite a bit. I made several updates and there is more to come. Highlights:

• First and foremost, I re-did some of the code regarding multithreading in the main() proc, using a list of threads and a join(). Similar changes to other areas are still in the works.
• Changed initialization code so that only selected channels are passed to the Channel_Config class (better and somewhat more efficient)
• Added a command line option to specify the location of the log-file. -L or --log_file
• Added timing info in the log-file (start time, end time, duration of run)
• Flushing (writing to disk) the log-file upon each call. Adds some overhead, but reduces chances that last log-entries get lost on abnormal termination.

To be done:

• Further improvements to multi-threading (most notably FetchData and derivatives)
• I added some debug-prints in the code. Handy, but should be removed from the 'Production-version'.
• Make validation of file locations specified in command line more robust (--log-file, but also --output, --cache)
• Intermediate flushes of the cache-file (if we decide we want them).

Hope this helps!

Best regards, Frans

[hikavdh]

I have to look at your threading changes further and thanks for the spelling corrections.:-) I'm thinking of replacing the ready flags with event objects with a 5 second timeout on the wait to look around. But about the other things.

• The flushing: I did not know the impact, but if it is so little...
• The loading of unused channels is intentional with a future graphic configuration in mind. Else the skipping of remarked lines can easier be done by changing parameter 3 (isremark) in the get_line function from None to False.
• The logging parameter is fine, but if you want to go that path, I thing you should also implement it as full configuration file parameter and add it to write_opts_to_log. Also you might then think about some schedule to keep more then just the last one.
• You debug log messages you should write through the InfoFiles class to detail_output or conditional on the write_info_files flag. That way you can leave them.

[hikavdh]

Oh, and one other thing. I don't want to make unneeded changes to the master branch. Improvements should go to tvgrab22 branch. As said logging and configuration have been updated there, but it's still work in progress. Hika

[hikavdh]

I copied part of your changes over. I left out the non loading of inactive channels and the log_file parameter. The later should go into the 22 branch. I suggest using one of the higher unused log_levels (1024) for debug messages. That way they can without problem stay in place. See https://github.com/tvgrabbers/tvgrabnlpy/releases/tag/beta-2.1.10-p20150808 And see for new Events as signal for the wait loops. https://github.com/tvgrabbers/tvgrabnlpy/releases/tag/beta-2.1.10-p20150809

Hika