search

lululxvi / deepxde

A library for scientific machine learning and physics-informed learning
https://deepxde.readthedocs.io
GNU Lesser General Public License v2.1
2.63k stars 739 forks source link

Support for Data Loader #1765

Closed dezenn closed 3 months ago

dezenn commented 4 months ago

Is there a way to use a DataGenerator like e.g. torch.utils.data.DataLoader? As far as I understand I have to initiate deepxde.Model(data, net) where data is my whole data set. But in the case of having big data this is not realizable and thus I want to train by loading data in batches.

praksharma commented 3 months ago

Generally, we don't have big data even if you are using DeepONets. You can load 1D numpy arrays (int 64) of shape 1 million on a normal PC and it will only consume 8 MB of RAM. This is more than enough for most applications in DeepXDE.