search

microsoft / microxcaling

PyTorch emulation library for Microscaling (MX)-compatible data formats
MIT License
164 stars 21 forks source link

TypeError: gelu() got an unexpected keyword argument 'approximate' #17

Closed Arthur-Ling closed 9 months ago

Arthur-Ling commented 9 months ago

When I try to use microxcaling with timm.swin_small, an error happens"TypeError: gelu() got an unexpected keyword argument 'approximate'". And my mx_specs ={ 'w_elem_format': 'fp6_e3m2', 'a_elem_format': 'fp6_e3m2', 'block_size': 32, 'bfloat': 16, 'custom_cuda': True, 'quantize_backprop': False, } mx_specs = finalize_mx_specs(mx_specs) mx_mapping.inject_pyt_ops(mx_specs)

what should I do?

rizhao-msft commented 9 months ago

I see the problem, we wrote the GELU stuff a while back. In older versions of pytorch there's no approximate kwarg. See link.

We'll push a fix soon. Can you paste here the line in your code where it calls GELU? This way we can be 100% sure the fix works.

Arthur-Ling commented 9 months ago

when I use timm.create_model('swin_small_patch4_window7_224'), the error happens

Arthur-Ling commented 9 months ago

"pytorch-cuda = 11.8 pytorch = 2.1.0 torchvision=0.16 torchaudio=2.1.0" is my pytorch version, is it too old?

rizhao-msft commented 9 months ago

Should be fixed by cd356dd10ae63880b475c0903c6b59d2bf4a7f8e.