search

wala / ML

Eclipse Public License 2.0
23 stars 17 forks source link

Initial tensor dataset support #123

Closed khatchad closed 6 months ago

khatchad commented 6 months ago

Basic fix for #89; more APIs to come.

khatchad commented 6 months ago

Sorry, may need to make some changes here.

khatchad commented 6 months ago

I think I'm hitting an unrelated bug here.

khatchad commented 6 months ago

I made it a bit more robust regarding the "for each" statement.

khatchad commented 6 months ago

Thanks, @msridhar. While this works, as I try to add more API, I realize that I've treated this particular API procedurally. It should return a Dataset object, which subsequent API calls can be invoked on. I'll most likely change some of this in the next PR to accommodate the Dataset object being returned.

khatchad commented 6 months ago

Thanks, @msridhar. While this works, as I try to add more API, I realize that I've treated this particular API procedurally. It should return a Dataset object, which subsequent API calls can be invoked on. I'll most likely change some of this in the next PR to accommodate the Dataset object being returned.

I've done that here: https://github.com/ponder-lab/ML/commit/c0bc8b25ba8e51403a5dd87f4edf08ed08bac7f7.

Now, I think we can add more methods to: https://github.com/ponder-lab/ML/commit/c0bc8b25ba8e51403a5dd87f4edf08ed08bac7f7#diff-679e457ba33ce5658ad9444fd41a8839e363bac4c0a268b27a86bbaad0c22056R340-R345.

Datasets are used in chains that apply transformations, and I think we can add the transforming methods there.

The tf2_test_dataset.py test works, but that commit is now failing on the newly added test which applies a shuffle() transformation: https://github.com/ponder-lab/ML/commit/1fc1d3df9ef4d50f0fc4e5aae54063307adc5329. I'll work on that now.