launching job with dbx launch --trace --job-output-log-level all prints notebook output periodically alongside job status output.
Current Behavior
When a line in the job output contains an emoji, the byte count offset appears to be incorrect, because the next time dbx prints job output, the first few characters are missing:
[dbx][2022-08-03 16:31:19.798] [Run Id: 167366] Latest cluster logs - [2022-08-01T12:25:52.588427][DatasetDeleter] ✅ finished deleting. Summary: {'seconds elapsed': 98.59}
[dbx][2022-08-03 16:31:24.684] [Run Id: 167366] Current run status info - result state: None, lifecycle state: RUNNING, state message: In run
[dbx][2022-08-03 16:31:44.680] [Run Id: 167366] Latest cluster logs - 022-08-03T16:31:32.661517] done. calculating statistics
now, when the new line also contains an emoji, the byte count is off by more characters. It appears to be 3 chars per emoji but I'm not sure, maybe 2.
Steps to Reproduce (for bugs)
Context
The current implementation reads from DataBricks API endpoint /jobs/runs/get-output and seeks forward an offset number of bytes into the string response (similar to 2.0/dbfs/read endpoint's offset request parameter):
Expected Behavior
launching job with
dbx launch --trace --job-output-log-level all
prints notebook output periodically alongside job status output.Current Behavior
When a line in the job output contains an emoji, the byte count offset appears to be incorrect, because the next time
dbx
prints job output, the first few characters are missing:specifically this is the error:
now, when the new line also contains an emoji, the byte count is off by more characters. It appears to be 3 chars per emoji but I'm not sure, maybe 2.
Steps to Reproduce (for bugs)
Context
The current implementation reads from DataBricks API endpoint
/jobs/runs/get-output
and seeks forward an offset number of bytes into the string response (similar to2.0/dbfs/read
endpoint'soffset
request parameter):I'm thinking if I change to UTF-16 encoding, the byte count would be accurate.
Your Environment