(I couldn't find anything related after an intensive web search)
I am facing a very confusing issue. I believe the issue started from mrjob v0.6.8 and persists till the latest version. Here is the sample code:
`
from mrjob.job import MRJob
from mrjob.step import MRStep
import sys
`
(Note: don't know why underscores are stripped from above main function declaration)
To run the code, need to create a local directory (e.g., test_input) and put a random small text file in it. When running from a (CentOS) Linux shell in a (Python 3.8.9) virtualenv that has mrjob 0.6.7 installed, it runs fine. However, the code crashes with the following exception when running in a virtualenv that has mrjob 0.6.8 (and beyond) installed (the command line: python test.py test_input):
File "test.bytes.py", line 8, in init_mergedrs
sys.stderr.write('Processing file.\n')
TypeError: a bytes-like object is required, not 'str'
If I comment out the sys.stderr.write (putting a 'pass' statement), the code still crashes locally, but the offending line is now in the 'print(header)' statement (same exception).
The code runs fine remotely on hadoop cluster though (with either mrjob 0.6.7 or 0.6.8 and beyond). Checking v0.6.8 change log doesn't reveal anything that gives any hint. Can anybody help? Thanks.
(The issue also happens in Python 3.7 and 3.9).
(I couldn't find anything related after an intensive web search) I am facing a very confusing issue. I believe the issue started from mrjob v0.6.8 and persists till the latest version. Here is the sample code: ` from mrjob.job import MRJob from mrjob.step import MRStep import sys
class MRSp(MRJob):
if name == 'main':
` (Note: don't know why underscores are stripped from above main function declaration) To run the code, need to create a local directory (e.g., test_input) and put a random small text file in it. When running from a (CentOS) Linux shell in a (Python 3.8.9) virtualenv that has mrjob 0.6.7 installed, it runs fine. However, the code crashes with the following exception when running in a virtualenv that has mrjob 0.6.8 (and beyond) installed (the command line: python test.py test_input): File "test.bytes.py", line 8, in init_mergedrs sys.stderr.write('Processing file.\n') TypeError: a bytes-like object is required, not 'str'
If I comment out the sys.stderr.write (putting a 'pass' statement), the code still crashes locally, but the offending line is now in the 'print(header)' statement (same exception).
The code runs fine remotely on hadoop cluster though (with either mrjob 0.6.7 or 0.6.8 and beyond). Checking v0.6.8 change log doesn't reveal anything that gives any hint. Can anybody help? Thanks. (The issue also happens in Python 3.7 and 3.9).