[Orca] Orca torch `data_creator` needs to support a custom dataloader

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc

Apache License 2.0

6.69k stars 1.26k forks source link

we haven't supportted customized dataloader yet, for issues like this we suggest you to overload dataloader like this:

import torch

class Prefetchtorchloader(PrefetchLoader):
    def __init__(self,
                 loader):
        self.loader = loader.loader
        self.mean = loader.mean
        self.std = loader.std
        self.fp16 = loader.fp16
        self.random_erasing = loader.random_erasing

        self.collate_fn = torch.utils.data.dataloader.default_collate

    def __iter__(self):
        for batch in self.loader:
            yield self.collate_fn(batch)

And we may plan to support iterator style dataloader in est. later.

intel-analytics / ipex-llm

[Orca] Orca torch `data_creator` needs to support a custom dataloader #6072