search

con / duct

A helper to run a command, capture stdout/stderr and details about running
MIT License
1 stars 1 forks source link

Make output UNBUFFERED for python children #61

Open yarikoptic opened 3 weeks ago

yarikoptic commented 3 weeks ago
It does help ```shell ❯ duct ./test_script.py --duration 1 --memory-size=1000 2>&1 | ts Jun 09 11:18:48 this is of test of STDERR Jun 09 11:18:50 this is of test of STDOUT Jun 09 11:18:50 Test completed. Consumed 1000 MB for 1 seconds with CPU load factor 10000. Jun 09 11:18:50 duct is executing ./test_script.py --duration 1 --memory-size=1000... Jun 09 11:18:50 Log files will be written to .duct/logs/2024.06.09T11.18.48-1439380_ Jun 09 11:18:50 Jun 09 11:18:50 Exit Code: 0 Jun 09 11:18:50 Command: ./test_script.py --duration 1 --memory-size=1000 Jun 09 11:18:50 Log files location: .duct/logs/2024.06.09T11.18.48-1439380_ Jun 09 11:18:50 Wall Clock Time: 1.749 sec Jun 09 11:18:50 Memory Peak Usage: 1.5% Jun 09 11:18:50 CPU Peak Usage: 103.0% ❯ PYTHONUNBUFFERED=1 duct ./test_script.py --duration 1 --memory-size=1000 2>&1 | ts Jun 09 11:18:54 duct is executing ./test_script.py --duration 1 --memory-size=1000... Jun 09 11:18:54 Log files will be written to .duct/logs/2024.06.09T11.18.54-1439428_ Jun 09 11:18:54 this is of test of STDOUT Jun 09 11:18:54 this is of test of STDERR Jun 09 11:18:56 Test completed. Consumed 1000 MB for 1 seconds with CPU load factor 10000. Jun 09 11:18:56 Jun 09 11:18:56 Exit Code: 0 Jun 09 11:18:56 Command: ./test_script.py --duration 1 --memory-size=1000 Jun 09 11:18:56 Log files location: .duct/logs/2024.06.09T11.18.54-1439428_ Jun 09 11:18:56 Wall Clock Time: 1.756 sec Jun 09 11:18:56 Memory Peak Usage: 1.5% Jun 09 11:18:56 CPU Peak Usage: 103.0% ```

and with this change:

❯ duct ./test_script.py --duration 1 --memory-size=1000 2>&1 | ts
Jun 09 11:28:55 this is of test of STDOUT
Jun 09 11:28:55 this is of test of STDERR
Jun 09 11:28:57 Test completed. Consumed 1000 MB for 1 seconds with CPU load factor 10000.
Jun 09 11:28:57 duct is executing ./test_script.py --duration 1 --memory-size=1000...
Jun 09 11:28:57 Log files will be written to .duct/logs/2024.06.09T11.28.55-1441799_
Jun 09 11:28:57 
Jun 09 11:28:57 Exit Code: 0
Jun 09 11:28:57 Command: ./test_script.py --duration 1 --memory-size=1000
Jun 09 11:28:57 Log files location: .duct/logs/2024.06.09T11.28.55-1441799_
Jun 09 11:28:57 Wall Clock Time: 1.931 sec
Jun 09 11:28:57 Memory Peak Usage: 1.5%
Jun 09 11:28:57 CPU Peak Usage: 104.0%

BUT I am not yet sure we should do that (by default at least), since it changes operating conditions for the process and in some cases people would not even care to receive "online" output.

asmacdo commented 3 weeks ago

Option 1: add --python-unbuffered -u, when passed it will set PYTHONUNBUFFERED

Modifying env vars has a smell to it. And this option is only for python programs.

Option 2: Documentation

asmacdo commented 3 weeks ago

Todo(asmacdo) see how 'watch' deals with captured outputs for when comparing differences

'watch -d'