How large memory is required for the experiment

lucidrains / lambda-networks

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

MIT License

1.53k stars 157 forks source link

How large memory is required for the experiment #1

Open amiltonwong opened 4 years ago

amiltonwong commented 4 years ago

Hi, @lucidrains ,

Thanks you a lot for releasing the prototype so quickly. How large memory is required for the experiment?

THX!

lucidrains commented 4 years ago

@amiltonwong thanks for your interest! I'll be trying it out next week in some generative models

have you tried it and found it to be memory intensive?

PistonY commented 4 years ago

I try out LambdaResnet50 with 64 batch_size about cost 9-10GB gpu memory in FP32 precision,it's much larger than Resnet50

jason90330 commented 4 years ago

Thanks you for your Implementation so efficientcy, but I have the similar problems to @PistonY . I use the lambda layer before I feed my image to the efficientNet-b4, and I would like to move my model into GPU but it took 18.76GB.

Here is the parameters in my lambda layer.

VuHoangvn commented 4 years ago

You should specify r

jason90330 commented 4 years ago

@VuHoangvn Thank a lot, I have revised the question I asked before, but I am still wondering if I can use the lambda layer into other SOTA model such as efficientnet. just like the image I post, can I insert this layer in front of my model directly? thank you. BTW, I have tested the score of the question I asked, but I got a lower score than without the lambda layer.