galipremsagar
commented
3 years ago @vuule This looks like a json reader/libcudf issue. Because I'm not able to repro this by creating a string column from cudf:
>>> import cudf
>>> cdf = cudf.read_json('sample.json', lines=True)
>>> import pandas as pd
>>> df = pd.read_json('sample.json', lines=True)
>>> df
value data
0 16 thirty_chars_plus_two_newlines\n\n
1 19 more_data_with_newline\r\n
>>> cdf
value data
0 16 thirty_chars_plus_two_newlines\n\n
1 19 more_data_with_newline\r\n
>>> cdf.data.str.len()
0 34
1 25
Name: data, dtype: int32
>>> s = cudf.Series(["thirty_chars_plus_two_newlines\n\n", "more_data_with_newline\r\n"])
>>> s
0 thirty_chars_plus_two_newlines\n\n
1 more_data_with_newline\r\n
dtype: object
>>> s[0]
'thirty_chars_plus_two_newlines\n\n'
>>> s.str.len()
0 32
1 24
dtype: int32
vuule
github-actions[bot]
GregoryKimball
Describe the bug Retrieving strings that contain a newline character from a Series returns a string with an "escaped" backslash (
\\n) instead of\n. The cudf output does not match the equivalent pandas output.Steps/Code to reproduce bug 1.) Create a JSON file with the following contents, called test.json:
2.) Create dataframes using both Pandas and cuDF, and compare the output of extracting a single element from a series:
Expected behavior The output should match pandas output.
Environment overview (please complete the following information)
Environment details
Additional context