search

tf-encrypted / moose

Secure distributed dataflow framework for encrypted machine learning and data processing
Apache License 2.0
58 stars 16 forks source link

[Not for merging] Support passing op name inside the kernels produced… #1015

Closed voronaam closed 2 years ago

voronaam commented 2 years ago

… by the AsyncSession

voronaam commented 2 years ago

I actually have a changeset where the task creation is moved to the AsyncExecutor. It was needed to create a task per a "slab" of operations. A chunk of ops delimited by ReceiveOp. This was from the time where we wanted to not do a tokio task per Op.

jvmncs commented 2 years ago

we don't seem to have a strong need for it right now

it would be nice to eventually be able to add some structure to the op names in the textual format, similar to what we saw with TF. for example, an edsl.add @ Replicated(alice, bob, carole) could be named add_0 in the logical level, but then expand during compilation to produce ops named e.g. add_0/alice/add_0, add_0/alice/add_1, add_0/bob/add_1, etc (actual naming here TBD). one concrete thing this would give us that we don't have currently is the ability to retain the positional order of outputs in pymoose

mortendahl commented 2 years ago

it would be nice to eventually be able to add some structure to the op names in the textual format

we could potentially keep track of this in the Session implementations