search

Microsemi / switchtec-kernel

A kernel module for the Microsemi PCIe switch
GNU General Public License v2.0
45 stars 31 forks source link

Inputs needed : failure to allocate coherent buffer #93

Open dusane opened 3 years ago

dusane commented 3 years ago

With 1.2rc3 driver i am trying to allocate 128 M of coherent memory using dma_alloc_coherent. which fails, even 64M allocation fails - at swiotlb buffer full.. (32 M works though). I have tried combos of following configs CMA configs (CONFIG_CMA_SIZE_MBYTES = 256, CONFIG_CMA_ALIGNMENT=12), high value of swiotlb=65535 in cmdline. tried explicitely setting dma_coherent_mask. But still i hit the failure. Any suggestion what i am missing ? This is with 4.4.36

lsgunth commented 3 years ago

No Idea. That seems like a crazy large amount of memory to expect to get out of the CMA. Not sure if it supports that. There may also be bugs in the ancient kernel.

Either way this isn't really a switchtec issue... You'll probably need to dig into the CMA code to figure out what is going on.

epilmore commented 3 years ago

Here are some settings we have used which have worked for large allocations.

CONFIG_CMA_AREAS=100

CONFIG_CMA_SIZE_MBYTES=4096

CONFIG_CMA_SIZE_SEL_MBYTES=y

CONFIG_CMA_ALIGNMENT=12

Did not change swiotlb boot parameter from default. Also, note that we were booting in 64-bit mode, which you might need to do for large allocations also.

Eric

On Wed, Oct 7, 2020 at 3:33 PM Manish Dusane notifications@github.com wrote:

With 1.2rc3 driver i am trying to allocate 128 M of coherent memory using dma_alloc_coherent. which fails, even 64M allocation fails - at swiotlb buffer full.. (32 M works though). I have tried combos of following configs CMA configs (CONFIG_CMA_SIZE_MBYTES = 256, CONFIG_CMA_ALIGNMENT=12), high value of swiotlb=65535 in cmdline. tried explicitely setting dma_coherent_mask. But still i hit the failure. Any suggestion what i am missing ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Microsemi/switchtec-kernel/issues/93, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHWQCZR66V22NDJ2QLZIACLSJTUD3ANCNFSM4SH7PFWQ .

-- Eric Pilmore epilmore@gigaio.com http://gigaio.com Phone: (858) 775 2514

This e-mail message is intended only for the individual(s) to whom it is addressed and may contain information that is privileged, confidential, proprietary, or otherwise exempt from disclosure under applicable law. If you believe you have received this message in error, please advise the sender by return e-mail and delete it from your mailbox. Thank you.

dusane commented 3 years ago

@lsgunth I agree its not really a switchtec issue. I just need inputs and advice from team. @epilmore Thanks Eric, will try these. I am booting in 64 bit mode too.

dusane commented 3 years ago

@epilmore These configs worked for me, Thanks. I am not completely sure of all its side effects though.