Sub streams implementation

[emil14]

Sub streams with support of unlimited nesting allows asynchronously process nested data but they are not implemented and it seems to be very hard to achieve.

Example from FBP book:

 < city1 
     < cust11 detl111 detl112...> 
     < cust12 detl111 detl112...>
  >
  < city2 
     < cust21 detl211 detl122...> 
     < cust22 detl221 detl222...>
  >
  ...

Problems

• Type-Checking: sub-streams represent not just (nested) lists but (also) hierarchic structures. It's not clear how to type-check such mechanics.
• Invalid bracketing - what to do (and how to implement such handling) if structure of sub-streams is invalid? e.g. not enought closing brackets or ordering of items is messed

---

Related to https://github.com/emil14/fbp-book-ru/blob/main/7_compos.md (sub-streams) and #58

Related to #58

---

Are staticly typed sub-streams even possible? Imagine a component with sub-stream sensitive inport - where does it takes input? From several different components that we hope sends their data in the right order?

[emil14]

Don't implement

Sub-streams represents structured data. It's possible to avoid them and use lists, structs, maps

Problems

• Sub-streams allows asynchronous processing of nested lists which is in the spirit of FBP

[emil14]

Bracket data type

Follow classical FBP and introduce brackets data type

Problems

• Union type (or some other type-checking mechanism) must be implemented
• Runtime "reflection" (messages must contain their types, not only values)
• Complex operator logic for sub-stream support (How FBP solves this?)
• Another one data type
• Brackets are not data in FBP but more like control flow primitives (so it's not 100% true FBP way)

[emil14]

Bools

There is already a type with 2 possible values - boolean. Just is it with union type:

F 5 4 3 T F 2 1 0 T -> + -> 3 12

Problems

• Same as with brackets unless new data type
• Won't work if sub-stream itself consists of booleans

[emil14]

Signals

Use one sig as open bracket and two as a closed (or vice versa)

() () 3 3 ()
() () 2 2 ()
  -> + -> 4 6

Problems

• Probably even more complicated to implement than brackets/bools because it assumes some internal counting inside operators
• Sig might be deleted (#63)

[emil14]

Only flat sub streams

Do not use sub-streams to represent nested structures, use them to represent dynamic lists only

Problems

• Can't represent real lists due to unlimited nesting of real lists in nature
• All the problems from https://github.com/emil14/neva/issues/35#issuecomment-1061258737 except operators mechanics must be somewhat simpler
• Sub-streams represent not only lists but also hierarchic structures

[emil14]

Native (struct-based) interfaces

Go has builtin interfaces like Reader or Closer. Brackets might be implemented in a similar way using structures

{ open: bool }

Problems

• Won't work if the data itself has form of {open:bool}. Especially in case of gradual typing

---

Gradual typing fix

The following form will make gradual typing problem less likely (although won't solve it completely)

{
  bracket: {}  // <- field and value signaling that message is a bracket
  open: bool
}

[emil14]

Group-extended struct-based

Modification of a prev variant but with group field. The meaning is that classical FBP's bracket-IPs can contain group-name, see book

{
  group string
  open bool
}

Problems

• Typing - it looks like group field assumes some dynamic type-checking
• Same as in the prev solution

[emil14]

Sub-stream defenitions

Just thinking out loud!

Maybe there's a way to have substreams descried staticly just as we have port-types and like we would like to have types? Something like a:

bank-stream:
    branch: 
        type: int
        children: # must be array because there could be stream like (a (b b) (c c))
            - account: 
                type: int
                children:
                    - date:
                        type: int
                    - trans:
                        type: str

Maybe key names even can be ommited somehow?

[emil14]

Generic struct

Implement StreamItem<T> generic type to operate on streams. This should solve "stream item is data OR bracket" typing problem.

{
  bracket: bool
  open: bool
  data: T | null 
}

Problems

• Optional type is still needed to represent T | null, so data | bracket problem just being replaced with another (maybe even worse) one
• Poor typing that allows to break invariant via { bracket: false, data: null } (maybe that's not the only one invalid value)