Modin Project Suggestions - 2021 Edition

devin-petersohn commented 3 years ago

This issue will serve as a pointer to potential larger projects that new or existing contributors can do to get involved in the development of Modin. This is not necessarily complete, nor do all of these need to be implemented by the end of the year.

See contributing guidelines here: https://modin.readthedocs.io/en/latest/contributing.html

Conversations and questions about individual projects should be directed to the issue for that specific project.

Execution backends

mpi4py
Python multiprocessing
Celery
Postgres

Modin dataframe enhancements

Query optimization
Data placement and shuffling optimization
Fine grained type inference/induction
Clean abstraction for Modin dataframe and its metadata
Partition size for better interactivity on dataframes of all sizes

Dataframe algebraic operator support

sort
join
explode
to_labels
from_labels
infer_types

APIs and UI elements

SQL API
Spreadsheet API
Progress bar
GeoPandas API

Fault tolerance and error handling

Logging
Fault tolerance for node or worker failures

pandas API operators

This will not be exhaustive, but a sample of the missing APIs are listed here. See the docs for a complete list.

DataFrame.infer_objects
DataFrame.align
DataFrame.explode
DataFrame.expand
DataFrame.interpolate
DataFrame.reindex_like

devin-petersohn commented 3 years ago

Please direct comments to the individual issues for the projects. If an issue does not exist, feel free to create one!

devin-petersohn commented 2 years ago

Retiring at the end of 2021

modin-project / modin