search

eschnett / IndexSpaces.jl

Mapping physics quantities onto hardware for efficient parallelization.
MIT License
2 stars 0 forks source link

Baseband beamformer should support s=sigma #1

Open kmsmith137 opened 1 year ago

kmsmith137 commented 1 year ago

Context: According to Erik in slack (#perimeter 22-12-31), "The code that rounds before shifting doesn't properly handle the corner case where the shift count is zero and it should add 1/2. instead, it would probably add INTMAX/2 or something silly. this could be easily fixed at the cost of a few additional instructions."

eschnett commented 1 year ago

In the current version (not yet passed on to Dustin), the kernel has an info output parameter that encodes an error condition. At startup, the shift count is checked, and if it cannot be handled, and error is reported. (s=3 is not yet handled since it would make the kernel more expensive.)