rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.04k stars 872 forks source link

[API]: "Private" (imported) Python APIs used in across other RAPIDS projects #15240

Open mroeschke opened 4 months ago

mroeschke commented 4 months ago

This is the result of a quick audit and does not include private APIs on public objects e.g. cudf.Series._column

TLDR observations:

  1. There seems to be a common desire to create a Column object (this might be solved with pylibcudf)?
  2. There seems to be a secondary desire to create a Buffer object

cuspatial

cudf.core.copy_types.BooleanMask (for typing) cudf.core.copy_types.GatherMap (for typing) cudf.core.column.ColumnBase cudf.core.column.ListColumn cudf.core.column.as_column cudf.core.column.column_empty cudf.core.buffer.acquire_spill_lock

from cudf._lib.column cimport Column, column from cudf._lib.cpp.column.column cimport column from cudf._lib.cpp.column.column_view cimport column_view from cudf._lib.cpp.table.table_view cimport table_view from cudf._lib.cpp.table.table cimport table from cudf._lib.utils cimport columns_from_table_view from cudf._lib.utils cimport columns_from_unique_ptr from cudf._lib.cpp.types cimport size_type from cudf._lib.utils cimport table_view_from_table

cugraph

cudf.core.buffer.as_buffer cudf.core.column.build_column cudf.core.column.column_empty cudf.core.column.as_column cudf.core.column.build_categorical_column cudf.core.column.ListColumn cudf._lib.transform.bools_to_mask

cuml

cudf.core.buffer.acquire_spill_lock cudf.core.buffer.Buffer cudf.utils.dtypes.min_signed_type

Morpheus

cudf.core.frame.Frame cudf.core.index._index_from_data cudf.core.subword_tokenizer.SubwordTokenizer

from cudf._lib.column cimport Column from cudf._lib.cpp.io.types cimport table_with_metadata from cudf._lib.cpp.table.table_view cimport table_view from cudf._lib.cpp.types cimport size_type from cudf._lib.utils cimport data_from_unique_ptr from cudf._lib.utils cimport get_column_names from cudf._lib.utils cimport table_view_from_table

Merlin

cudf.tests.utils.assert_eq

cuopt

cudf.core.buffer.as_buffer cudf.core.column.build_column

bdice commented 4 months ago

Almost everything from _lib (which also means "almost everything using cudf Cython via cimport") will have a mirror image in pylibcudf. In the short term I would focus efforts on the uses of cudf.core, and let cudf._lib be resolved at a later time as pylibcudf matures.

mroeschke commented 4 months ago

Organizing non-_lib usage by "theme":

Column APIs

cudf.core.column.build_column cudf.core.column.column_empty cudf.core.column.as_column cudf.core.column.build_categorical_column cudf.core.column.ListColumn

Buffer APIs

cudf.core.buffer.as_buffer cudf.core.buffer.acquire_spill_lock cudf.core.buffer.Buffer

Other APis

cudf.core.frame.Frame cudf.core.index._index_from_data cudf.core.subword_tokenizer.SubwordTokenizer cudf.utils.dtypes.min_signed_type cudf.tests.utils.assert_eq