search

wala / ML

Eclipse Public License 2.0
25 stars 17 forks source link

Missing callbacks related to datasets #113

Open khatchad opened 10 months ago

khatchad commented 10 months ago

Missing experimental_distribute_datasets_from_function

Called here: https://github.com/mead-ml/mead-baseline/blob/5d7632bb151c2d09501ebf49f36ba8c4204df4c8/mead/api_examples/pretrain_discrim_tf.py#L307

The callback function is dataset_test_fn, defined here: https://github.com/mead-ml/mead-baseline/blob/master/mead/api_examples/pretrain_discrim_tf.py#L301-L306

But that function doesn't show up in the call graph. The method reference should be: < PythonLoader, Lscript pretrain_discrim_tf.py/get_dataset, do()LRoot; >. However, we don't see it in the call graph nodes: https://gist.github.com/khatchad/bd48384f3561fdcc7c8c6e9141d93de5

Related to #92 and blocked on #89.

khatchad commented 7 months ago

We might be at the point where we can now fix this if we can treat distributed values as datasets.