Added retry on process exec for unicode issues

SeanTConrad commented 1 month ago

Why are these changes needed?

Another user had this issue in their logs:

2024-07-31 16:47:29 Gathering source code statistics for: hostaways.git
2024-07-31 16:48:24 std_exec Error: 'utf-8' codec can't decode byte 0x8b in position 77953: invalid start byte, ['git', 'show', "--format='%G?'", '457cb08fa518e95f5749f5bbd86bc97ab620d55e']
2024-07-31 16:48:24 std_exec Error: 'utf-8' codec can't decode byte 0x8b in position 78112: invalid start byte, ['git', 'show', '457cb08fa518e95f5749f5bbd86bc97ab620d55e']
2024-07-31 16:48:26 std_exec Error: 'utf-8' codec can't decode byte 0x8b in position 20608: invalid start byte, ['git', 'show', "--format='%G?'", '7d104d11d790f5f798731f7865274997872b94a6']
2024-07-31 16:48:26 std_exec Error: 'utf-8' codec can't decode byte 0x8b in position 20790: invalid start byte, ['git', 'show', '7d104d11d790f5f798731f7865274997872b94a6']

I haven't thought of a way to reproduce, so it's tough to test. I think it's a Windows' carriage return in a commit message.

Related issue number

Checks

[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ x ] I've made sure all auto checks have passed.

Summary by Sourcery

This pull request addresses an issue where the std_exec function fails to decode output due to Unicode errors. A retry mechanism has been added to handle these errors by re-executing the command with error replacement, improving robustness in handling encoding issues.

Bug Fixes:
- Added a retry mechanism in the std_exec function to handle UnicodeDecodeError by attempting to re-execute the command with error replacement for encoding issues.

sourcery-ai[bot] commented 1 month ago

Reviewer's Guide by Sourcery

This pull request addresses an issue with Unicode decoding errors during the execution of subprocess commands. The implementation adds a retry mechanism that attempts to re-execute the command with error handling for Unicode issues.

File-Level Changes

Files	Changes
`src/verinfast/utils/utils.py`	Introduced a retry mechanism in the std_exec function to handle UnicodeDecodeError by re-executing the command with shell=True and 'replace' error handling in the decode method.

Tips

- Trigger a new Sourcery review by commenting `@sourcery-ai review` on the pull request. - Continue your discussion with Sourcery by replying directly to review comments. - You can change your review settings at any time by accessing your [dashboard](https://app.sourcery.ai): - Enable or disable the Sourcery-generated pull request summary or reviewer's guide; - Change the review language; - You can always [contact us](mailto:support@sourcery.ai) if you have any questions or feedback.

SeanTConrad commented 1 month ago

Why not just do the replace in the first call.

I'm not super familiar with what "replace" does, so I was trying not to impact the "happy path." If you think the "replace" is safe, I can collapse/ do it in the first call.

SeanTConrad commented 1 month ago

Why not just do the replace in the first call.

I went ahead and changed it.

VerinFast / verinfast