move deeplake dataset to _orig_dataset to enable ld.dataset variable …

activesoull commented 1 month ago

…for integration like mmdet

🚀 🚀 Pull Request

Impact

[ x] Bug fix (non-breaking change which fixes expected existing functionality)
[ ] Enhancement/New feature (adds functionality without impacting existing logic)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

fixed the issue related to the enterprise dataloader's dataset property overwriting, as the integrations are using that property in the pipelines

Description

Things to be aware of

Things to worry about

Additional Context

activesoull commented 1 month ago

This was the change which introduced broken shuffling. We reverted it to fix the shuffling. Can you please explain what is this doing?

The issue was that our mmdet integration requires that the dataloader should have the dataloader.dataset property. So the training pipeline will get from the dataloader the dataset MMdetDataset and extract the properties in train/val/test loops moved all the dataset related logics into the internal _orig_dataset , property and we reserve the dataset property for the training frameworks.

so the workflow looks like this

we create dataloader
set the dataloader's dataset property to be MMdetDataset here
and the dataloader was failing here and all the pleas where we wanted to access self.datset as if was changed from deeplake dataset to MMdetDataset object.

will add test case in indra to keep the bug track on shuffling quality check