Fix/faster code

[marikaris]

• Replace downloading consensus history with parsing the consensus history file
• Replace formats and + with f strings
• Write consensus per 1000 lines instead of each line
• Show progress in consensus file generation per 1000 lines instead of for each line

[SietsmaRJ]

Globally: I see that a lot of string formatting has been replaced by F string formatting, this does force a user to use Python 3.7 or greater. Is this intentional?

I also see a lot of csv and tsv formatting in strings, which is not all that fast in Python. This isn't much of an issue if only ran twice each year, but upgrading to Pandas might be considered for even more speed. (If of course worth the investment.)

[marikaris]

Globally: I see that a lot of string formatting has been replaced by F string formatting, this does force a user to use Python 3.7 or greater. Is this intentional?

I also see a lot of csv and tsv formatting in strings, which is not all that fast in Python. This isn't much of an issue if only ran twice each year, but upgrading to Pandas might be considered for even more speed. (If of course worth the investment.)

Yes, the f-strings were intentionally because they are much faster than string formatting and they were easy to fix.

I agree that pandas would be great for the job and it would make the code a lot faster. I simply didn't have the time to refactor that much (and I don't have much experience with pandas, so it takes me a lot of time to get adjusted to again), but it's definitely something I want to do in the future.

[sonarcloud[bot]]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information