numaproj / numaflow-python

Numaflow Python SDK
Apache License 2.0
53 stars 18 forks source link

fix: time unit bug/dockerfile/add a guide #44

Closed jy4096 closed 1 year ago

jy4096 commented 1 year ago

numaflow uses millisecond, while the python sdk code uses second.

fix the time unit inconsistency bug. fix the dockerfile in flatmap example. add a developer guide.. let me know if we want to put it elsewhere.

Test

tested using the developer_guide.

Pipeline

apiVersion: numaflow.numaproj.io/v1alpha1
kind: Pipeline
metadata:
  name: pynumasdk
spec:
  vertices:
    - name: in
      source:
        # A self data generating source
        generator:
          rpu: 10
          duration: 1s
    - name: cat
      udf:
        container:
          # compute the sum
          image: quay.io/numaio/numaflow-python/map-forward-message:latest
    - name: counter
      udf:
        container:
          # compute the sum
          image: quay.io/numaio/numaflow-python/reduce-counter:latest
        groupBy:
          window:
            fixed:
              length: 10s
          keyed: true
    - name: sink
      scale:
        min: 1
      sink:
        udsink:
          container:
            image: quay.io/numaio/numaflow-python/sink-log:latest
  edges:
    - from: in
      to: cat
    - from: cat
      to: counter
      parallelism: 1
    - from: counter
      to: sink

Result

image image
codecov[bot] commented 1 year ago

Codecov Report

Merging #44 (78e1779) into main (d74d063) will not change coverage. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #44   +/-   ##
=======================================
  Coverage   98.53%   98.53%           
=======================================
  Files           9        9           
  Lines         273      273           
  Branches       21       21           
=======================================
  Hits          269      269           
  Misses          2        2           
  Partials        2        2           
Impacted Files Coverage Δ
pynumaflow/function/server.py 97.18% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more