arg "num_workers" invalid

[SouthMemory]

Dear renato

class BcolzDataBunch(DataBunch):
    @classmethod
    def create(cls, train_ds, valid_ds, **kwargs):
        if 'num_workers' in kwargs: kwargs.pop('num_workers')
        return super().create(train_ds, valid_ds, num_workers=0, **kwargs)

    def normalize(self, do_x=True, do_y=False):
        if getattr(self,'norm',False): raise Exception('Can not call normalize twice')
        self.norm = partial(normalize_fun, do_x=do_x, do_y=do_y)
        self.denorm = denormalize
        self.add_tfm(self.norm)
        return self

here if 'num_workers' in kwargs: kwargs.pop('num_workers') make arg "num_workers" invalid, Is there a reason?

Can I free it ?

[renato145]

It's because of how bcolz works, if you use more than 1 worker it will get stuck. I don't know if there is a way to get bcolz to work on multiple workers.