search

Texera / texera

Collaborative Machine-Learning-Centric Data Analytics Using Workflows
https://texera.github.io
Apache License 2.0
162 stars 73 forks source link

Orleans.Activities evaluation #650

Closed shengquan-ni closed 6 years ago

shengquan-ni commented 6 years ago

evaluations of the project on https://github.com/OrleansContrib/Orleans.Activities

shengquan-ni commented 6 years ago

Open Questions

1. How are operators created when it gets a workflow?

For now, I don't think Orleans.Activities will create different operators when it gets a workflow. It simply parses the workflow file, creates a WorkflowInstance and saves it in a WorkflowGrain when client manually creates that grain.

2. How do grains (operators) pass information to the next grain?

For now, I don't think Orleans.Activities will pass information between pairs of WorkflowGrains. Every WorkflowGrain is an entity with the whole workflow. Information passing is limited between client and WorkflowGrain only.

3. What is Activity class (Orleans.Activities)?

Orleans.Activites is the name of the project package. It is not a part from Orleans. The Activity class is actually from System.Activities There is no connection between Activity and Orleans before the author creates Orleans.Activites

4. What is WorkflowGrain? How is it different from a normal grain?

It is derived from Grain with a persistent state in Orleans, it just acts like a normal grain and maintains a WorkflowState as its persistent state, there is also a WorkflowHost inside itself.

5. What is WorkflowState?

It is a class defined by the author, it maintains a dictionary that maps System.Xml.Linq.XName to System.Runtime.DurableInstancing.InstanceValue, which saves variables in the workflow. It also maintains a System.Activities.WorkflowIdentity at the same time.`

6. When is the WorkflowGrain constructor called?

Grains do have constructors, and it will be called before OnActivateAsync() see also: Sequence of events in a grain's life cycle

7. How is OnUnhandledExceptionAsync used?

When a unhandled exception occurs inside the workflow during execution, the WorkflowInstance inside the WorkflowHost of the WorkflowGrain will notice that and propagate that exception out until it reaches the WorkflowGrain.

8. What is WorkflowControl?

It is a WorkflowHost which encapsulates the operations towards a WorkflowInstance. By assigning StartingAsync and CompletedAsync to a lambda function, we can have the control before and after the workflow execution.

9. What is CreateObjectReference (Program.cs line 93)

To subscribe to a notification, the client must first create a local C# object that implements the observer interface. It then calls a static method on the observer factory, CreateObjectReference(), to turn the C# object into a grain reference, which can then be passed to the subscription method on the notifying grain.

see also: Client Obsevers

10. "Idempotent forward recovery" (Program.cs line 102)

Since the WorkflowGrain is designed as "one-shot" grain, It will execute the workflow only once.

In a specific scenario:

  1. I create a WorkflowGrain A in the client by using GetGrain.
  2. I call a method on WorkflowGrain A with some arguments, eventually, this method will pass my arguments to the workflow and execute that workflow inside the grain (calling System.Activities.Hosting.Controller.Run). (i.e. A.SomeMethod(arg1,arg2))
  3. I get the result from WorkflowGrain A after a while. (i.e I get 5)
  4. I call the method again on WorkflowGrain A with some different arguments. (i.e. A.SomeMethod(arg3,arg4))
  5. I get the same result from WorkflowGrain A immediately. (i.e I get 5 again)

11. Why does Multiply do RunAsync() and Add do RunToCompletionAsync()

There are 2 ways to execute a workflow:

Use RunAsync() when the workflow doesn't start with accepting an incoming operation. It will not block the grain.

Use RunToCompletionAsync() only for short running workflows, when the workflow doesn't start with accepting an incoming operation and completes without calling back to the caller. It will block the grain.

The author added a Delay of 10 seconds in MultiplyActivity, so it is not a short running workflow. Thus, Multiply uses RunAsync() and Add uses RunToCompletionAsync().

12. Is Workflow Foundation really single-threaded?

Yes, by default.

See also:

13. Can we pause a WorkflowInstance?

Yes, but there is no such a thing in Olreans.Activities. This project didn't expose the APIs for pause/resume. See also: Pausing and Resuming a Workflow

shengquan-ni commented 6 years ago

Workflow Foundation Functionalities

Pause/Resume

Comprehensive explaination: Pausing and Resuming a Workflow

Tracking and Tracing

Workflow tracking provides insight into the execution of a workflow instance. The tracking events are emitted from a workflow at the workflow instance level and when activities within the workflow execute. A workflow tracking participant needs to be added to the workflow host to subscribe to tracking records. The tracking records are filtered using a tracking profile. The .Net Framework provides an ETW (Event Tracing for Windows) tracking participant, and a basic profile is installed in the machine.config file.

This feature allow users to receive messages and extract data from the workflow when start/finish an operator in that workflow, or when start/pause/resume/complete the entire workflow.

Sample output: Sample output

Extensions

Users are able to design their custom opertors by inheriting a C# class from System.Activities.NativeActivity and creating a XAML for it, then use it as a widget in Workflow Foundation to get it visualized.