dma_map_sgtable internally uses the dma_map_sg_attrs helper. The problem is that dma_map_sg_attrs returns the number of entries mapped, whereas dma_map_sgtable returns nonzero on failure. This leads to dma_map_sgtable returning non-zero-but-positive values which tricks other areas of the stack into thinking nents is a valid pointer.
This checks if nents is valid and returns zero if so, updating the nents field in sgt. This fixes PRIME render offload with nvidia-drm.
This review handles installs before FreeBSD version 1400066, where dma_map_sgtable comes from drm-kmod instead of from base.
dma_map_sgtable internally uses the dma_map_sg_attrs helper. The problem is that dma_map_sg_attrs returns the number of entries mapped, whereas dma_map_sgtable returns nonzero on failure. This leads to dma_map_sgtable returning non-zero-but-positive values which tricks other areas of the stack into thinking nents is a valid pointer.
This checks if nents is valid and returns zero if so, updating the nents field in sgt. This fixes PRIME render offload with nvidia-drm.
This review handles installs before FreeBSD version 1400066, where dma_map_sgtable comes from drm-kmod instead of from base.
Related: https://reviews.freebsd.org/D37611