friendly-traceback / friendly

Friendly-traceback's version used in most situations
https://friendly-traceback.github.io/docs/index.html
MIT License
40 stars 5 forks source link

Writing the conf atomically to allow use in parallel environments. #25

Closed JulienPalard closed 2 years ago

JulienPalard commented 2 years ago

It may be just tightly related to my usage: I run HackInScience unit test in parallel because ... it's 16× faster to run them in parallel on my machine.

But each test use friendly, and having 16 friendly in parallel mess with the config file, rendering it unparsable, leading to my tests failing.

A good way to show the issue out of friendly:

from multiprocessing import Pool

def f(x):
    with open("testfile.txt", "w") as fd:
        for i in range(10):
            fd.write(str(x))
            fd.write("\n")

with Pool(16) as p:
    p.map(f, range(150))

leading to things like:

86
86
86
86
86
86
86
86
86
86
6
146
146

Without the patch my test are failing like 9 times every 10. With the path I haven't seen them fail (tried ~10 times)

sourcery-ai[bot] commented 2 years ago

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 0.79%.

Quality metrics Before After Change
Complexity 4.89 ⭐ 4.67 ⭐ -0.22 👍
Method Length 56.12 ⭐ 51.22 ⭐ -4.90 👍
Working memory 6.58 🙂 6.67 🙂 0.09 👎
Quality 76.18% 76.97% 0.79% 👍
Other metrics Before After Change
Lines 125 134 9
Changed files Quality Before Quality After Quality Change
friendly/settings.py 76.18% ⭐ 76.97% ⭐ 0.79% 👍

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation

Legend and Explanation

The emojis denote the absolute quality of the code:

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.


Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

aroberge commented 2 years ago

Thank you for this contribution, as it might be useful for people who, like you, are working in environments that use friendly in parallel sessions.

However, ... Do you have any reason to use friendly instead of friendly-traceback? Since you do your own formatting of the information with HackInScience, you should only need to use friendly-traceback. friendly is now "just" a fancy formatter which pulls the information from friendly-traceback, like HackInScience does, and uses its own method and various heavy dependencies such as Rich and pygments, to do some formatting.

In addition, friendly-traceback does not save settings like friendly does, and you would never have noticed this problem [and provided a nice solution for me ;-) ] if you would have use friendly-traceback instead.

JulienPalard commented 2 years ago

I'm using friendly-traceback 99% of the time, but sometimes instead of running a student function I run a whole student script (for exericses like my-add or calculator1), like this:

correction_helper.py-    try:
correction_helper.py-        proc = subprocess.run(
correction_helper.py-            [
correction_helper.py-                "python3",
correction_helper.py-                "-m",
correction_helper.py:                "friendly",
correction_helper.py-                "--formatter",
correction_helper.py:                "correction_helper.friendly_traceback_markdown",
correction_helper.py-                file,
correction_helper.py-                *args,
correction_helper.py-            ],
correction_helper.py-            stdout=subprocess.PIPE,
correction_helper.py-            stderr=subprocess.PIPE,

Am I using your libs right? Or do you see a better way?

aroberge commented 2 years ago

You should be able to replace "friendly" by "friendly_traceback" and it should work. I only have unit tests for friendly_traceback and I do have a test (that you initially wrote, I believe) for users of custom formatters.

aroberge commented 2 years ago

I will have to revert this pull request as it raises an exception when I run locally:

FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\Andre\\AppData\\Local\\FriendlyTraceback\\tmpy2l_iv9z' -> 'C:\\Users\\Andre\\AppData\\Local\\FriendlyTraceback\\friendly.ini'

friendly_traceback does not save settings file (friendly.ini) and can be used from the command line instead of friendly.