measure the execution time of once pipeline, and change numbers of specified processor for next time

[lain1936]

Sorry my english is not very good , but i am working on it.

When needing to more than once (or repeate ) to call some pipelines, it seem to pre compile to define counts of consumer is not good enough.

Suppose many agents connected to a chatroom or game AOI(Area of Interest) , and push/receive messages very frequently. We maybe use some frp pattern (Functional reactive programming) to change related states in one tick of the timewheel. for eg. use this module, https://github.com/Qqwy/elixir_gen_frp/blob/master/lib/gen_frp/callback.ex#L34 to regist a interval callback ( a Flowex.Pipeline start function for eg.) at a certain timeout

It's time sensitive , so we need to limit the max execution time of one sequence computational task.

When get the option's interval of outside, we can to retry to spawn more processor in Flowex.Pipeline， so It need some ways to dynamically change the number of specified pipe.

  def start(pipeline_module, opts) do
    if opts.muti and opts.sup_pid  do     
       do_change(sup_pid, pipeline_module, opts)
    else
        {:ok, sup_pid} = Flowex.Supervisor.start_link(pipeline_module)
        do_start(sup_pid, pipeline_module, opts)
    end
  end

pipe :add_one,  name: fn ->String.to_atom("Flowex_ModuleName_PrevToNextLayerOrIndexNumber_add_one") end
pipe :mult_by_two, name: fn ->String.to_atom("Flowex_ModuleName_LayerOrIndexNumber_mult_by_two") end, min_time:30, step_by: 2, max_number:10

Something like this interface, in every tick we calculate the time difference (in microseconds) of begin and end of a Pipeline, if less than limit, it is ok, if not, in next tick we add more number of specified Pipe by step. (we also need limit max number of a pipe to avoid too much process)

[antonmi]

Hi, @lain1936 ! Thanks for sharing your idea!