search

deJQK / FracBits

Neural Network Quantization With Fractional Bit-widths
11 stars 5 forks source link

Need help with pyarrow configuration #2

Open Akimoto-Cris opened 2 years ago

Akimoto-Cris commented 2 years ago

Hi @deJQK,

Seems like you are using apache arrow for distributed training. Can you explain more on how to configurate environment for pyarrow? I cannot start training as i always get FileNotFoundError: [Errno 2] No such file or directory: 'hadoop'

Thanks in advance.

deJQK commented 2 years ago

Hi @Akimoto-Cris , thanks for your interests in our work. It seems that we use pyarrow during data loader for ImageNet with lmdb. If possible, could you try to avoid using lmdb but directly use the vanilla dataloader to load images? Hope it helps.

Akimoto-Cris commented 2 years ago

Thanks @deJQK. Your works are always inspiring for the community.

I change to original loader. But the problem is still there. I will try get rid of the ultron_io in the code.