vuule
commented
3 years ago @galipremsagar I believe this is a feature request. Currently CSV reader and writer don't support categorical data, AFAIK.
Closed galipremsagar closed 3 years ago
vuule
commented
3 years ago @galipremsagar I believe this is a feature request. Currently CSV reader and writer don't support categorical data, AFAIK.
kkraus14
commented
3 years ago As a stopgap from the Python side we could materialize the dictionary encoded column before writing the CSV.
github-actions[bot]
commented
3 years ago This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.
github-actions[bot]
commented
3 years ago This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d.
beckernick
commented
3 years ago This has been implemented, likely in in https://github.com/rapidsai/cudf/pull/6829 . Closing
import cudf
import pandas as pd
pdf = pd.DataFrame()
pdf['a'] = pd.Series(['dsfdsfsdf','sdfsdfsdf','sdfsdfsdfs','sdfsdfsdf'], dtype='category')
gdf = cudf.from_pandas(pdf)
print(pdf.to_csv())
print(gdf.to_csv())
,a
0,dsfdsfsdf
1,sdfsdfsdf
2,sdfsdfsdfs
3,sdfsdfsdf
,a
0,dsfdsfsdf
1,sdfsdfsdf
2,sdfsdfsdfs
3,sdfsdfsdf
Describe the bug When there is a categorical column in the dataframe, csv writer is returning the code instead of the actual categorical data.
Steps/Code to reproduce bug
Expected behavior Categorical data should be returned instead of codes.
Environment overview (please complete the following information)
Environment details Please run and paste the output of the
cudf/print_env.shscript here, to gather any other relevant environment detailsClick here to see environment details
**git*** commit 7bea2645ca4bfd9db344e7f84bc61438749770e3 (HEAD -> branch-0.17, upstream/branch-0.17) Author: Conor Hoekstra <36027403+codereport@users.noreply.github.com> Date: Tue Nov 3 03:56:49 2020 -0500 Implement `cudf::round` floating point and integer types (`HALF_UP`) (#6562) **git submodules*** ***OS Information*** DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS" NAME="Ubuntu" VERSION="20.04.1 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.1 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal Linux pgali-HP-Z8-G4-Workstation 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux ***GPU Information*** Tue Nov 3 18:02:45 2020 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Quadro RTX 8000 Off | 00000000:22:00.0 Off | Off | | 33% 30C P8 4W / 260W | 2132MiB / 48601MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 1 Quadro RTX 8000 Off | 00000000:2D:00.0 On | Off | | 33% 34C P8 22W / 260W | 351MiB / 48592MiB | 18% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 963 G /usr/lib/xorg/Xorg 4MiB | | 0 N/A N/A 1833 G /usr/lib/xorg/Xorg 4MiB | | 0 N/A N/A 5362 C python 531MiB | | 0 N/A N/A 6150 C python 529MiB | | 0 N/A N/A 7208 C python 529MiB | | 0 N/A N/A 7343 C .../envs/cudf_dev/bin/python 529MiB | | 1 N/A N/A 963 G /usr/lib/xorg/Xorg 39MiB | | 1 N/A N/A 1833 G /usr/lib/xorg/Xorg 153MiB | | 1 N/A N/A 1960 G /usr/bin/gnome-shell 146MiB | +-----------------------------------------------------------------------------+ ***CPU*** Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 46 bits physical, 48 bits virtual CPU(s): 12 On-line CPU(s) list: 0-11 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Gold 6128 CPU @ 3.40GHz Stepping: 4 CPU MHz: 1200.477 CPU max MHz: 3700.0000 CPU min MHz: 1200.0000 BogoMIPS: 6800.00 Virtualization: VT-x L1d cache: 192 KiB L1i cache: 192 KiB L2 cache: 6 MiB L3 cache: 19.3 MiB NUMA node0 CPU(s): 0-11 Vulnerability Itlb multihit: KVM: Vulnerable Vulnerability L1tf: Mitigation; PTE Inversion Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d ***CMake*** /home/pgali/anaconda3/envs/cudf_dev/bin/cmake cmake version 3.18.2 CMake suite maintained and supported by Kitware (kitware.com/cmake). ***g++*** /usr/bin/g++ g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ***nvcc*** /usr/local/cuda/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Wed_Jul_22_19:09:09_PDT_2020 Cuda compilation tools, release 11.0, V11.0.221 Build cuda_11.0_bu.TC445_37.28845127_0 ***Python*** /home/pgali/anaconda3/envs/cudf_dev/bin/python Python 3.7.8 ***Environment Variables*** PATH : /home/pgali/anaconda3/envs/cudf_dev/bin:/home/pgali/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda/bin LD_LIBRARY_PATH : :/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64 NUMBAPRO_NVVM : NUMBAPRO_LIBDEVICE : CONDA_PREFIX : /home/pgali/anaconda3/envs/cudf_dev PYTHON_PATH : ***conda packages*** /home/pgali/anaconda3/condabin/conda # packages in environment at /home/pgali/anaconda3/envs/cudf_dev: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge abseil-cpp 20200225.2 he1b5a44_2 conda-forge alabaster 0.7.12 py_0 conda-forge apipkg 1.5 pypi_0 pypi appdirs 1.4.3 py_1 conda-forge argon2-cffi 20.1.0 py37h8f50634_2 conda-forge arrow-cpp 1.0.1 py37hd1d2865_14_cuda conda-forge arrow-cpp-proc 1.0.1 cuda conda-forge async_generator 1.10 py_0 conda-forge attrs 20.2.0 pyh9f0ad1d_0 conda-forge aws-c-common 0.4.59 he1b5a44_0 conda-forge aws-c-event-stream 0.1.6 h84e28f3_5 conda-forge aws-checksums 0.1.9 he252421_2 conda-forge aws-sdk-cpp 1.8.63 h9b98462_0 conda-forge babel 2.8.0 py_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.1 py_0 conda-forge black 19.10b0 py_4 conda-forge bleach 3.2.1 pyh9f0ad1d_0 conda-forge bokeh 2.2.2 py37hc8dfbb8_0 conda-forge boost-cpp 1.74.0 h9359b55_0 conda-forge brotli 1.0.9 he1b5a44_2 conda-forge brotlipy 0.7.0 py37hb5d75c8_1001 conda-forge bzip2 1.0.8 h516909a_3 conda-forge c-ares 1.16.1 h516909a_3 conda-forge ca-certificates 2020.6.20 hecda079_0 conda-forge certifi 2020.6.20 py37he5f6b98_2 conda-forge cffi 1.14.3 py37h00ebd2e_1 conda-forge cfgv 3.2.0 py_0 conda-forge chardet 3.0.4 py37he5f6b98_1008 conda-forge clang 8.0.1 hc9558a2_2 conda-forge clang-tools 8.0.1 hc9558a2_2 conda-forge clangxx 8.0.1 2 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge cloudpickle 1.6.0 py_0 conda-forge cmake 3.18.2 h3f3948e_0 conda-forge cmake_setuptools 0.1.3 py_0 rapidsai commonmark 0.9.1 py_0 conda-forge cryptography 3.1.1 py37hff6837a_1 conda-forge cudatoolkit 11.0.221 h6bb024c_0 nvidia cudf 0.17.0a0+92.gf552adaeb3 pypi_0 pypi cudnn 8.0.0 cuda11.0_0 nvidia cupy 7.8.0 py37h0ce7dbb_0 rapidsai cython 0.29.21 py37hb892b2f_1 conda-forge cytoolz 0.11.0 py37h8f50634_1 conda-forge dask 2.30.0+16.gfbe51746 pypi_0 pypi dask-cudf 0.17.0a0+249.g979dc78c98.dirty pypi_0 pypi decorator 4.4.2 py_0 conda-forge defusedxml 0.6.0 py_0 conda-forge distlib 0.3.1 pyh9f0ad1d_0 conda-forge distributed 2.30.0+7.gdcb46d07 pypi_0 pypi dlpack 0.3 he1b5a44_1 conda-forge docutils 0.16 py37he5f6b98_2 conda-forge double-conversion 3.1.5 he1b5a44_2 conda-forge editdistance 0.5.3 py37h3340039_2 conda-forge entrypoints 0.3 py37hc8dfbb8_1002 conda-forge execnet 1.7.1 pypi_0 pypi expat 2.2.9 he1b5a44_2 conda-forge fastavro 1.0.0.post1 py37h8f50634_1 conda-forge fastrlock 0.5 py37h3340039_1 conda-forge filelock 3.0.12 pyh9f0ad1d_0 conda-forge flake8 3.8.3 py_1 conda-forge flatbuffers 1.12.0 he1b5a44_0 conda-forge freetype 2.10.3 he06d7ca_0 conda-forge fsspec 0.8.4 py_0 conda-forge future 0.18.2 py37hc8dfbb8_2 conda-forge gflags 2.2.2 he1b5a44_1004 conda-forge glog 0.4.0 h49b9bf7_3 conda-forge gmp 6.2.0 he1b5a44_3 conda-forge grpc-cpp 1.32.0 h7997a97_1 conda-forge heapdict 1.0.1 py_0 conda-forge hypothesis 5.37.3 py_0 conda-forge icu 67.1 he1b5a44_0 conda-forge identify 1.5.6 pyh9f0ad1d_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge imagesize 1.2.0 py_0 conda-forge importlib-metadata 2.0.0 py_1 conda-forge importlib_metadata 2.0.0 1 conda-forge iniconfig 1.1.0 pyh9f0ad1d_0 conda-forge ipykernel 5.3.4 py37hc6149b9_1 conda-forge ipython 7.18.1 py37hc6149b9_1 conda-forge ipython_genutils 0.2.0 py_1 conda-forge isort 5.0.7 py37hc8dfbb8_0 conda-forge jedi 0.17.2 py37hc8dfbb8_1 conda-forge jinja2 2.11.2 pyh9f0ad1d_0 conda-forge jpeg 9d h516909a_0 conda-forge jsonschema 3.2.0 py_2 conda-forge jupyter_client 6.1.7 py_0 conda-forge jupyter_core 4.6.3 py37hc8dfbb8_2 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge krb5 1.17.1 hfafb76e_3 conda-forge lcms2 2.11 hbd6801e_0 conda-forge ld_impl_linux-64 2.35 h769bd43_9 conda-forge libblas 3.8.0 17_openblas conda-forge libcblas 3.8.0 17_openblas conda-forge libcurl 7.71.1 hcdd3856_8 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 hcdb4288_3 conda-forge libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 9.3.0 h5dbcf3e_17 conda-forge libgfortran-ng 9.3.0 he4bcb1c_17 conda-forge libgfortran5 9.3.0 he4bcb1c_17 conda-forge libgomp 9.3.0 h5dbcf3e_17 conda-forge liblapack 3.8.0 17_openblas conda-forge libllvm10 10.0.1 he513fc3_3 conda-forge libllvm8 8.0.1 hc9558a2_0 conda-forge libnghttp2 1.41.0 h8cfc5f6_2 conda-forge libopenblas 0.3.10 pthreads_h4812303_5 conda-forge libpng 1.6.37 hed695b0_2 conda-forge libprotobuf 3.13.0.1 h8b12597_0 conda-forge librmm 0.17.0a201018 cuda11.0_g8cdd176_21 rapidsai-nightly libsodium 1.0.18 h516909a_1 conda-forge libssh2 1.9.0 hab1572f_5 conda-forge libstdcxx-ng 9.3.0 h2ae2ef3_17 conda-forge libthrift 0.13.0 h5aa387f_6 conda-forge libtiff 4.1.0 hc7e4089_6 conda-forge libutf8proc 2.5.0 h516909a_2 conda-forge libuv 1.40.0 hd18ef5c_0 conda-forge libwebp-base 1.1.0 h516909a_3 conda-forge llvmlite 0.34.0 py37h5202443_2 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.2 he1b5a44_3 conda-forge markdown 3.3.1 pyh9f0ad1d_0 conda-forge markupsafe 1.1.1 py37hb5d75c8_2 conda-forge mccabe 0.6.1 py_1 conda-forge mimesis 4.0.0 pyh9f0ad1d_0 conda-forge mistune 0.8.4 py37h8f50634_1002 conda-forge more-itertools 8.5.0 py_0 conda-forge msgpack-python 1.0.0 py37h99015e2_2 conda-forge nbclient 0.5.1 py_0 conda-forge nbconvert 6.0.7 py37hc8dfbb8_1 conda-forge nbformat 5.0.8 py_0 conda-forge nbsphinx 0.7.1 pyh9f0ad1d_0 conda-forge nccl 2.7.8.1 h4962215_100 nvidia ncurses 6.2 he1b5a44_2 conda-forge nest-asyncio 1.4.1 py_0 conda-forge nodeenv 1.5.0 pyh9f0ad1d_0 conda-forge notebook 6.1.4 py37hc8dfbb8_1 conda-forge numba 0.51.2 py37h9fdb41a_0 conda-forge numpy 1.19.2 py37h7ea13bd_1 conda-forge numpydoc 1.1.0 py_1 conda-forge nvtx 0.2.1 py37h8f50634_2 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openssl 1.1.1h h516909a_0 conda-forge orc 1.6.5 hd3605a7_0 conda-forge packaging 20.4 pyh9f0ad1d_0 conda-forge pandas 1.1.3 py37h9fdb41a_2 conda-forge pandavro 1.5.2 pypi_0 pypi pandoc 1.19.2 0 conda-forge pandocfilters 1.4.2 py_1 conda-forge parquet-cpp 1.5.1 2 conda-forge parso 0.7.1 pyh9f0ad1d_0 conda-forge partd 1.1.0 py_0 conda-forge pathspec 0.8.0 pyh9f0ad1d_0 conda-forge pexpect 4.8.0 pyh9f0ad1d_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.0.0 py37h718be6c_0 conda-forge pip 20.2.4 py_0 conda-forge pluggy 0.13.1 py37hc8dfbb8_3 conda-forge pre-commit 2.7.1 py37hc8dfbb8_1 conda-forge pre_commit 2.7.1 1 conda-forge prometheus_client 0.8.0 pyh9f0ad1d_0 conda-forge prompt-toolkit 3.0.8 py_0 conda-forge psutil 5.7.2 py37hb5d75c8_1 conda-forge ptyprocess 0.6.0 py_1001 conda-forge py 1.9.0 pyh9f0ad1d_0 conda-forge py-cpuinfo 7.0.0 pyh9f0ad1d_0 conda-forge pyarrow 1.0.1 py37h5b20ac3_14_cuda conda-forge pybind11 2.5.0 pypi_0 pypi pycodestyle 2.6.0 pyh9f0ad1d_0 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pyflakes 2.2.0 pyh9f0ad1d_0 conda-forge pygments 2.7.1 py_0 conda-forge pyopenssl 19.1.0 py_1 conda-forge pyorc 0.3.0 pypi_0 pypi pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyrsistent 0.17.3 py37h8f50634_1 conda-forge pysocks 1.7.1 py37he5f6b98_2 conda-forge pytest 6.1.1 py37hc8dfbb8_1 conda-forge pytest-benchmark 3.2.3 pyh9f0ad1d_0 conda-forge pytest-forked 1.3.0 pypi_0 pypi pytest-xdist 2.1.0 pypi_0 pypi python 3.7.8 h6f2ec95_1_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.7 1_cp37m conda-forge pytz 2020.1 pyh9f0ad1d_0 conda-forge pyyaml 5.3.1 py37hb5d75c8_1 conda-forge pyzmq 19.0.2 py37hac76be4_2 conda-forge rapidjson 1.1.0 he1b5a44_1002 conda-forge re2 2020.10.01 he1b5a44_0 conda-forge readline 8.0 he28a2e2_2 conda-forge recommonmark 0.6.0 py_0 conda-forge regex 2020.10.15 py37h8f50634_0 conda-forge requests 2.24.0 pyh9f0ad1d_0 conda-forge rhash 1.3.6 h516909a_1001 conda-forge rmm 0.17.0a201017 cuda_11.0_py37_g8cdd176_21 rapidsai-nightly send2trash 1.5.0 py_0 conda-forge setuptools 49.6.0 py37he5f6b98_2 conda-forge six 1.15.0 pyh9f0ad1d_0 conda-forge snappy 1.1.8 he1b5a44_3 conda-forge snowballstemmer 2.0.0 py_0 conda-forge sortedcontainers 2.2.2 pyh9f0ad1d_0 conda-forge spdlog 1.7.0 hc9558a2_2 conda-forge sphinx 3.2.1 py_0 conda-forge sphinx-copybutton 0.3.0 pyh9f0ad1d_0 conda-forge sphinx-markdown-tables 0.0.14 pyh9f0ad1d_1 conda-forge sphinx_rtd_theme 0.5.0 pyh9f0ad1d_0 conda-forge sphinxcontrib-applehelp 1.0.2 py_0 conda-forge sphinxcontrib-devhelp 1.0.2 py_0 conda-forge sphinxcontrib-htmlhelp 1.0.3 py_0 conda-forge sphinxcontrib-jsmath 1.0.1 py_0 conda-forge sphinxcontrib-qthelp 1.0.3 py_0 conda-forge sphinxcontrib-serializinghtml 1.1.4 py_0 conda-forge sphinxcontrib-websupport 1.2.4 pyh9f0ad1d_0 conda-forge sqlite 3.33.0 h4cf870e_1 conda-forge streamz 0.6.0 pypi_0 pypi tblib 1.6.0 py_0 conda-forge terminado 0.9.1 py37hc8dfbb8_1 conda-forge testpath 0.4.4 py_0 conda-forge tk 8.6.10 hed695b0_1 conda-forge toml 0.10.1 pyh9f0ad1d_0 conda-forge toolz 0.11.1 py_0 conda-forge tornado 6.0.4 py37h8f50634_2 conda-forge traitlets 5.0.5 py_0 conda-forge typed-ast 1.4.1 py37h516909a_0 conda-forge typing_extensions 3.7.4.3 py_0 conda-forge urllib3 1.25.10 py_0 conda-forge virtualenv 20.0.35 py37hc8dfbb8_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.35.1 pyh9f0ad1d_0 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge zeromq 4.3.3 he1b5a44_2 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.3.1 py_0 conda-forge zlib 1.2.11 h516909a_1010 conda-forge zstd 1.4.5 h6597ccf_2 conda-forgeAdditional context Surfaced while running fuzz tests: #6001