Closed peterdudfield closed 1 month ago
Yeh this should be part of creating the batch saving script which is mentioned in #7. Although currently the equivalent script is in PVNet so maybe it should happen over there?
We also already have the collate function we need to stack the samples together. Its here in datapipes
oh thanks, ill use that collate function
Ill ahve a go, as i need it for https://github.com/openclimatefix/pvnet_app/issues/116, and it seemed better to sit here
Deiceded not to do this, as the code is very simepl
from ocf_datapipes.batch.merge_numpy_examples_to_batch import stack_np_examples_into_batch
from torch.utils.data import DataLoader
DataLoader(
dataset,
batch_size=batch_size,
num_workers=num_workers,
pin_memory=True,
collate_fn=stack_np_examples_into_batch,
**kwargs,
)
Detailed Description
Add a torch dataloader
Context
Possible Implementation