numaproj / numaflow-python

Numaflow Python SDK
Apache License 2.0
52 stars 17 forks source link

Streamline API naming for `Server*` / `*Callable` #168

Open th0ger opened 4 months ago

th0ger commented 4 months ago

I just tried to get an overview of the server classes fingerprints (pynumaflow 0.7.0);

module class_name arg_name arg_type
pynumaflow.mapper MapServer mapper_instance MapSyncCallable
pynumaflow.mapper MapAsyncServer mapper_instance MapAsyncCallable
pynumaflow.mapper MapMultiprocServer mapper_instance MapSyncCallable
pynumaflow.reducer ReduceAsyncServer reducer_handler ReduceCallable
pynumaflow.mapstreamer MapStreamAsyncServer map_stream_instance MapStreamCallable
pynumaflow.sourcetransformer SourceTransformServer source_transform_instance SourceTransformCallable
pynumaflow.sourcetransformer SourceTransformMultiProcServer source_transform_instance SourceTransformCallable
pynumaflow.sourcer SourceServer sourcer_instance SourceCallable
pynumaflow.sourcer SourceAsyncServer sourcer_instance SourceCallable
pynumaflow.sinker SinkServer sinker_instance SyncSinkCallable
pynumaflow.sinker SinkAsyncServer sinker_instance AsyncSinkCallable
pynumaflow.sideinput SideInputServer side_input_instance RetrieverCallable

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

kohlisid commented 4 months ago

@th0ger Thanks for pointing this out! 1) For the reducer_handler stands out, suggesting rename reducer_instance --> This we can change for sure 2) Consider dropping Sync syntax altogether, ---> For this we might be considering this in the next iteration during refactoring, I want to make the type checking stricter for servers which support multiple types (sync, async, etc) so that we have better handling of inconsistencies 3) AsyncSinkCallable stands out to MapSyncCallable Agreed, will change this

th0ger commented 4 months ago

You're welcome, thanks for considering this. These are of course just nice-to-have breaking changes that you should wait and bundle with other API changes you plan.