OpenCL implementation of SimpleIntersectionFinder

JLBegin commented 2 years ago

Main Changes

Hardware accelerated intersections using the simple approach (no space partitioning).
Fully integrated with OpenCL photon propagation.
Properly works with the InteractionKey-based Logger (=> Stats.report() shows the same values as CPU version).
Between 100x and 500x faster (GTX960M) compared to CPU FastIntersectionFinder (KD-Tree), depending on scene complexity.
Added exampleSimple and exampleComplex to test performance.
Also includes #79

Limitations

Logger size slows down the code and is currently set a lot bigger than what is really needed. This will be partly solved with @PyMarc2 's batching feature. The other step will be to define optimal batch logger size based on a propagation test of a few photons (1k). Fixed with #79
In high-polygon scenes, the kernel execution time is usually the bottleneck due to intersection calculations, whereas in low-polygon scenes, the conversion from OpenCL logger data to the InteractionKey-based Logger takes the most time.

Example 1

exampleSimple.py: Low polygon scene.

N = 500k photons
 ... 1.183 s. [Build]
 ... 0.342 s. [Kernel execution]
 ... 1.170 s. [CL logger copy]
 ... 2.271 s. [Transfer to scene logger]
>>> (5.133 s) Total

Example 2

exampleComplex.py: High polygon scene (1.3k polygons) with LOW scattering

N = 500k photons
 ... 0.805 s. [Build]
 ... 5.253 s. [Kernel execution]
 ... 0.384 s. [CL logger copy]
 ... 1.106 s. [Transfer to scene logger]
>>> (7.614 s.)

Screenshot from 2022-11-17 12-02-05

Example 3

exampleComplex.py: High polygon scene (1.3k polygons) with HIGH scattering and CLPhotons.avgInteractions = 400

N = 50k photons
 ... 0.568 s. [Build]
 ... 1.644 s. [Kernel execution]
 ... 0.086 s. [CL logger copy]
 ... 0.579 s. [Transfer to scene logger]
>>> (2.939 s.)

Example 4

Test in very high polygon scene (500k polygons) The data related to the scene still only takes around 25 MB on GPU.

N = 10k photons
 ... 6.338 s. [Build]
 ... 3.463 s. [Kernel execution]
 ... 0.023 s. [CL logger copy]
 ... 1.265 s. [Transfer to scene logger]
>>> (16.035 s.)

JLBegin commented 1 year ago

Finally managed to fix bad intersections (until proven otherwise...).

One of the fixes that I used is to turn off normal smoothing for rays almost parallel to the surface. This is sub optimal and I will try to find a better way (like properly repositioning the photons on the surface represented by the smoothed normal instead of the mesh surface).

JLBegin commented 1 year ago

Sometimes I get an INVALID_COMMAND_QUEUE during the execution of the last batch when there is only a few photons left. Not sure why yet. But as an example, when propagating 500k photons for a fixed logger size (1e8) and work units (10k), it always works if I set the Kernel size under 260k photons, and always fails if I set the kernel size over 300k photons (this condition yields more photons at risk of not propagating fully during their batch time since batch logger memory is limiting). @PyMarc2

dccote commented 1 year ago

Where are we on this? Near completion? Branch is getting old

dccote commented 1 year ago

I am seeing many errors on macOS.

Traceback (most recent call last):
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 128, in _autoSetNWorkUnits
    optimalNWorkUnits = computeOptimalNWorkUnits()
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/testWorkUnits.py", line 46, in computeOptimalNWorkUnits
    source.propagate(scene, logger=logger, showProgress=False)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 35, in propagate
    IPP = self._getAverageInteractionsPerPhoton(scene)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 62, in _getAverageInteractionsPerPhoton
    self._measureIPP(scene)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 79, in _measureIPP
    self._propagateOpenCL(estimatedIPP, scene, tempLogger, showProgress=False)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 99, in _propagateOpenCL
    self._photons.propagate(IPP=IPP, verbose=showProgress)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLPhotons.py", line 60, in propagate
    program.launchKernel(kernelName="propagate", N=np.int32(params.workItemAmount),
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLProgram.py", line 30, in launchKernel
    self._build(CLObjects)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLProgram.py", line 59, in _build
    self._program = cl.Program(self._context, sourceCode).build()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyopencl/__init__.py", line 539, in build
    self._prg, was_cached = self._build_and_catch_errors(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyopencl/__init__.py", line 587, in _build_and_catch_errors
    raise err
pyopencl._cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz' on 'Apple' at 0xffffffff>:

<program source>:61:6: warning: no previous prototype for function 'wangHash'
uint wangHash(uint seed){
     ^
<program source>:70:7: warning: no previous prototype for function 'getRandomFloatValue'
float getRandomFloatValue(__global unsigned int *seeds, unsigned int id){
      ^
<program source>:85:6: warning: no previous prototype for function 'normalizeVectorLocal'
void normalizeVectorLocal(float3 *vector){
     ^
<program source>:94:6: warning: no previous prototype for function 'normalizeVectorGlobal'
void normalizeVectorGlobal(__global float3 *vector){
     ^
<program source>:103:6: warning: no previous prototype for function 'rotateAroundAxisGlobal'
void rotateAroundAxisGlobal(__global float3 *mainVector, __global float3 *axisVector, float theta){
     ^
<program source>:128:6: warning: no previous prototype for function 'rotateAroundAxisLocal'
void rotateAroundAxisLocal(float3 *mainVector, float3 *axisVector, float theta){
     ^
<program source>:153:6: warning: no previous prototype for function 'rotateAround'
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
     ^
<program source>:178:8: warning: no previous prototype for function 'getAnyOrthogonalGlobal'
float3 getAnyOrthogonalGlobal(__global float3 *vector){
       ^
<program source>:185:8: warning: no previous prototype for function 'getAnyOrthogonal'
float3 getAnyOrthogonal(float3 *vector){
       ^
<program source>:210:7: warning: no previous prototype for function 'getScatteringDistance'
float getScatteringDistance(float mu_t, float randomNumber){
      ^
<program source>:214:7: warning: no previous prototype for function 'getScatteringAnglePhi'
float getScatteringAnglePhi(float randomNumber){
      ^
<program source>:219:7: warning: no previous prototype for function 'getScatteringAngleTheta'
float getScatteringAngleTheta(float g, float randomNumber){
      ^
<program source>:229:18: warning: no previous prototype for function 'getScatteringAngles'
ScatteringAngles getScatteringAngles(float rndPhi, float rndTheta,__global Photon *photons,
                 ^
<program source>:255:13: error: global variables must have a constant address space qualifier
const float EPS = 0.00001f;
            ^
<program source>:256:13: error: global variables must have a constant address space qualifier
const float EPS_CORRECTION = 0.0005f;
            ^
<program source>:257:13: error: global variables must have a constant address space qualifier
const float EPS_PARALLEL = 0.00001f;
            ^
<program source>:258:13: error: global variables must have a constant address space qualifier
const float EPS_SIDE = 0.000001f;
            ^
<program source>:300:21: warning: no previous prototype for function '_getBBoxIntersection'
GemsBoxIntersection _getBBoxIntersection(Ray ray, float3 minCornerVector, float3 maxCornerVector) {
                    ^
<program source>:370:6: warning: no previous prototype for function '_findBBoxIntersectingSolids'
void _findBBoxIntersectingSolids(Ray ray, Scene *scene, uint gid){
     ^
<program source>:387:6: warning: no previous prototype for function '_sortSolidCandidates'
void _sortSolidCandidates(Scene *scene, uint gid) {
     ^
<program source>:412:10: warning: no previous prototype for function '_getTriangleIntersection'
HitPoint _getTriangleIntersection(Ray ray, float3 v1, float3 v2, float3 v3) {
         ^
<program source>:460:14: warning: no previous prototype for function '_findClosestPolygonIntersection'
Intersection _findClosestPolygonIntersection(Ray ray, uint solidID,
             ^
<program source>:489:7: warning: no previous prototype for function '_cotangent'
float _cotangent(float3 v0, float3 v1, float3 v2) {
      ^
<program source>:495:6: warning: no previous prototype for function 'setSmoothNormal'
void setSmoothNormal(Intersection *intersection, __global Triangle *triangles, __global Vertex *vertices) {
     ^
<program source>:517:6: warning: no previous prototype for function '_composeIntersection'
void _composeIntersection(Intersection *intersection, Ray *ray, Scene *scene) {
     ^
<program source>:528:14: warning: no previous prototype for function 'findIntersection'
Intersection findIntersection(Ray ray, Scene *scene, uint gid) {
             ^
<program source>:586:7: warning: no previous prototype for function '_getReflectionCoefficient'
float _getReflectionCoefficient(float n1, float n2, float thetaIn) {
      ^
<program source>:613:6: warning: no previous prototype for function '_getIsReflected'
bool _getIsReflected(float nIn, float nOut, float thetaIn, __global uint *seeds, uint gid) {
     ^
<program source>:622:7: warning: no previous prototype for function '_getReflectionDeflection'
float _getReflectionDeflection(float thetaIn) {
      ^
<program source>:626:7: warning: no previous prototype for function '_getRefractionDeflection'
float _getRefractionDeflection(float nIn, float nOut, float thetaIn) {
      ^
<program source>:632:6: warning: no previous prototype for function '_createFresnelIntersection'
void _createFresnelIntersection(FresnelIntersection* fresnelIntersection,
     ^
<program source>:643:21: warning: no previous prototype for function 'computeFresnelIntersection'
FresnelIntersection computeFresnelIntersection(float3 rayDirection, Intersection *intersection,
                    ^
<program source>:677:11: error: global variables must have a constant address space qualifier
const int NO_SOLID_ID = -1;
          ^
<program source>:678:11: error: global variables must have a constant address space qualifier
const int NO_SURFACE_ID = -1;
          ^
<program source>:680:6: warning: no previous prototype for function 'moveBy'
void moveBy(float distance, __global Photon *photons, uint photonID){
     ^
<program source>:684:6: warning: no previous prototype for function 'scatterBy'
void scatterBy(float phi, float theta, __global Photon *photons, uint photonID){
     ^
<program source>:689:6: warning: no previous prototype for function 'decreaseWeightBy'
void decreaseWeightBy(float delta_weight, __global Photon *photons, uint photonID){
     ^
<program source>:693:6: warning: no previous prototype for function 'interact'
void interact(__global Photon *photons, __constant Material *materials, __global DataPoint *logger,
     ^
<program source>:705:6: warning: no previous prototype for function 'scatter'
void scatter(__global Photon *photons, __constant Material *materials, __global uint *seeds, __global DataPoint *logger,
     ^
<program source>:717:6: warning: no previous prototype for function 'roulette'
void roulette(float weightThreshold, __global Photon *photons, __global uint *seeds, uint gid, uint photonID){
     ^
<program source>:730:6: warning: no previous prototype for function 'reflect'
void reflect(FresnelIntersection *fresnelIntersection, __global Photon *photons, uint photonID){
     ^
<program source>:731:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
                          ^
<program source>:734:6: warning: no previous prototype for function 'refract'
void refract(FresnelIntersection *fresnelIntersection, __global Photon *photons, uint photonID){
     ^
<program source>:735:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
                          ^
<program source>:738:6: warning: no previous prototype for function 'logIntersection'
void logIntersection(Intersection *intersection, __global Photon *photons, __global Surface *surfaces,
     ^
<program source>:766:7: warning: no previous prototype for function 'reflectOrRefract'
float reflectOrRefract(Intersection *intersection, __global Photon *photons, __constant Material *materials,
      ^
<program source>:772:30: warning: comparison of integers of different signs: 'int' and '__global uint' (aka '__global unsigned int')
    if (solidIDTowardsNormal != photons[photonID].solidID) {
        ~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:810:7: warning: no previous prototype for function 'propagateStep'
float propagateStep(float distance, __global Photon *photons, __constant Material *materials, Scene *scene,
      ^
<program source>:838:38: warning: comparison of integers of different signs: 'int' and '__global uint' (aka '__global unsigned int')
            if (solidIDTowardsNormal != photons[photonID].solidID) {
                ~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~

(options: -I /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyopencl/cl)
(source saved as /var/folders/vw/xyjt8vln3yb962l7lzwbgvrc0000gn/T/tmpj4dgkn0z.cl)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/exampleSimple.py", line 14, in <module>
    source = DivergentSource(position=Vector(0, 0, -0.1), direction=Vector(0, 0, 1), N=N,
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 227, in __init__
    super().__init__(position=position, direction=direction, diameter=diameter, N=N,
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 169, in __init__
    super().__init__(position=position, N=N, useHardwareAcceleration=useHardwareAcceleration)
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 26, in __init__
    useHardwareAcceleration = validateOpenCL()
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/__init__.py", line 15, in validateOpenCL
    CONFIG.validate()
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 58, in validate
    self._autoSetNWorkUnits()
  File "/Users/dccote/GitHub/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 134, in _autoSetNWorkUnits
    raise ValueError(f"The automatic test for optimal N_WORK_UNITS failed. Please manually run the test "
ValueError: The automatic test for optimal N_WORK_UNITS failed. Please manually run the test script 'testWorkUnits.py' and set N_WORK_UNITS in the config file at 'rayscattering/opencl/config.json'. 
... Error message: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz' on 'Apple' at 0xffffffff>:

<program source>:61:6: warning: no previous prototype for function 'wangHash'
uint wangHash(uint seed){
     ^
<program source>:70:7: warning: no previous prototype for function 'getRandomFloatValue'
float getRandomFloatValue(__global unsigned int *seeds, unsigned int id){
      ^
<program source>:85:6: warning: no previous prototype for function 'normalizeVectorLocal'
void normalizeVectorLocal(float3 *vector){
     ^
<program source>:94:6: warning: no previous prototype for function 'normalizeVectorGlobal'
void normalizeVectorGlobal(__global float3 *vector){
     ^
<program source>:103:6: warning: no previous prototype for function 'rotateAroundAxisGlobal'
void rotateAroundAxisGlobal(__global float3 *mainVector, __global float3 *axisVector, float theta){
     ^
<program source>:128:6: warning: no previous prototype for function 'rotateAroundAxisLocal'
void rotateAroundAxisLocal(float3 *mainVector, float3 *axisVector, float theta){
     ^
<program source>:153:6: warning: no previous prototype for function 'rotateAround'
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
     ^
<program source>:178:8: warning: no previous prototype for function 'getAnyOrthogonalGlobal'
float3 getAnyOrthogonalGlobal(__global float3 *vector){
       ^
<program source>:185:8: warning: no previous prototype for function 'getAnyOrthogonal'
float3 getAnyOrthogonal(float3 *vector){
       ^
<program source>:210:7: warning: no previous prototype for function 'getScatteringDistance'
float getScatteringDistance(float mu_t, float randomNumber){
      ^
<program source>:214:7: warning: no previous prototype for function 'getScatteringAnglePhi'
float getScatteringAnglePhi(float randomNumber){
      ^
<program source>:219:7: warning: no previous prototype for function 'getScatteringAngleTheta'
float getScatteringAngleTheta(float g, float randomNumber){
      ^
<program source>:229:18: warning: no previous prototype for function 'getScatteringAngles'
ScatteringAngles getScatteringAngles(float rndPhi, float rndTheta,__global Photon *photons,
                 ^
<program source>:255:13: error: global variables must have a constant address space qualifier
const float EPS = 0.00001f;
            ^
<program source>:256:13: error: global variables must have a constant address space qualifier
const float EPS_CORRECTION = 0.0005f;
            ^
<program source>:257:13: error: global variables must have a constant address space qualifier
const float EPS_PARALLEL = 0.00001f;
            ^
<program source>:258:13: error: global variables must have a constant address space qualifier
const float EPS_SIDE = 0.000001f;
            ^
<program source>:300:21: warning: no previous prototype for function '_getBBoxIntersection'
GemsBoxIntersection _getBBoxIntersection(Ray ray, float3 minCornerVector, float3 maxCornerVector) {
                    ^
<program source>:370:6: warning: no previous prototype for function '_findBBoxIntersectingSolids'
void _findBBoxIntersectingSolids(Ray ray, Scene *scene, uint gid){
     ^
<program source>:387:6: warning: no previous prototype for function '_sortSolidCandidates'
void _sortSolidCandidates(Scene *scene, uint gid) {
     ^
<program source>:412:10: warning: no previous prototype for function '_getTriangleIntersection'
HitPoint _getTriangleIntersection(Ray ray, float3 v1, float3 v2, float3 v3) {
         ^
<program source>:460:14: warning: no previous prototype for function '_findClosestPolygonIntersection'
Intersection _findClosestPolygonIntersection(Ray ray, uint solidID,
             ^
<program source>:489:7: warning: no previous prototype for function '_cotangent'
float _cotangent(float3 v0, float3 v1, float3 v2) {
      ^
<program source>:495:6: warning: no previous prototype for function 'setSmoothNormal'
void setSmoothNormal(Intersection *intersection, __global Triangle *triangles, __global Vertex *vertices) {
     ^
<program source>:517:6: warning: no previous prototype for function '_composeIntersection'
void _composeIntersection(Intersection *intersection, Ray *ray, Scene *scene) {
     ^
<program source>:528:14: warning: no previous prototype for function 'findIntersection'
Intersection findIntersection(Ray ray, Scene *scene, uint gid) {
             ^
<program source>:586:7: warning: no previous prototype for function '_getReflectionCoefficient'
float _getReflectionCoefficient(float n1, float n2, float thetaIn) {
      ^
<program source>:613:6: warning: no previous prototype for function '_getIsReflected'
bool _getIsReflected(float nIn, float nOut, float thetaIn, __global uint *seeds, uint gid) {
     ^
<program source>:622:7: warning: no previous prototype for function '_getReflectionDeflection'
float _getReflectionDeflection(float thetaIn) {
      ^
<program source>:626:7: warning: no previous prototype for function '_getRefractionDeflection'
float _getRefractionDeflection(float nIn, float nOut, float thetaIn) {
      ^
<program source>:632:6: warning: no previous prototype for function '_createFresnelIntersection'
void _createFresnelIntersection(FresnelIntersection* fresnelIntersection,
     ^
<program source>:643:21: warning: no previous prototype for function 'computeFresnelIntersection'
FresnelIntersection computeFresnelIntersection(float3 rayDirection, Intersection *intersection,
                    ^
<program source>:677:11: error: global variables must have a constant address space qualifier
const int NO_SOLID_ID = -1;
          ^
<program source>:678:11: error: global variables must have a constant address space qualifier
const int NO_SURFACE_ID = -1;
          ^
<program source>:680:6: warning: no previous prototype for function 'moveBy'
void moveBy(float distance, __global Photon *photons, uint photonID){
     ^
<program source>:684:6: warning: no previous prototype for function 'scatterBy'
void scatterBy(float phi, float theta, __global Photon *photons, uint photonID){
     ^
<program source>:689:6: warning: no previous prototype for function 'decreaseWeightBy'
void decreaseWeightBy(float delta_weight, __global Photon *photons, uint photonID){
     ^
<program source>:693:6: warning: no previous prototype for function 'interact'
void interact(__global Photon *photons, __constant Material *materials, __global DataPoint *logger,
     ^
<program source>:705:6: warning: no previous prototype for function 'scatter'
void scatter(__global Photon *photons, __constant Material *materials, __global uint *seeds, __global DataPoint *logger,
     ^
<program source>:717:6: warning: no previous prototype for function 'roulette'
void roulette(float weightThreshold, __global Photon *photons, __global uint *seeds, uint gid, uint photonID){
     ^
<program source>:730:6: warning: no previous prototype for function 'reflect'
void reflect(FresnelIntersection *fresnelIntersection, __global Photon *photons, uint photonID){
     ^
<program source>:731:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
                          ^
<program source>:734:6: warning: no previous prototype for function 'refract'
void refract(FresnelIntersection *fresnelIntersection, __global Photon *photons, uint photonID){
     ^
<program source>:735:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector, float3 *axisVector, float theta){
                          ^
<program source>:738:6: warning: no previous prototype for function 'logIntersection'
void logIntersection(Intersection *intersection, __global Photon *photons, __global Surface *surfaces,
     ^
<program source>:766:7: warning: no previous prototype for function 'reflectOrRefract'
float reflectOrRefract(Intersection *intersection, __global Photon *photons, __constant Material *materials,
      ^
<program source>:772:30: warning: comparison of integers of different signs: 'int' and '__global uint' (aka '__global unsigned int')
    if (solidIDTowardsNormal != photons[photonID].solidID) {
        ~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:810:7: warning: no previous prototype for function 'propagateStep'
float propagateStep(float distance, __global Photon *photons, __constant Material *materials, Scene *scene,
      ^
<program source>:838:38: warning: comparison of integers of different signs: 'int' and '__global uint' (aka '__global unsigned int')
            if (solidIDTowardsNormal != photons[photonID].solidID) {
                ~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~

(options: -I /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyopencl/cl)
(source saved as /var/folders/vw/xyjt8vln3yb962l7lzwbgvrc0000gn/T/tmpj4dgkn0z.cl)

PyMarc2 commented 1 year ago

Will test and confirm on macos/windows and linux tonight

PyMarc2 commented 1 year ago

Example.py was run on Montery 12.4 and failed.

Press enter to run the test to find the optimal value for N_WORK_UNITS, or enter it manually if you already know it: 
... Running a test to find optimal N_WORK_UNITS between 128 and 32768. This may take a few minutes. 
WARNING: Could not find the average interactions per photon (IPP) for this experiment. 
... [Estimating IPP]
Traceback (most recent call last):
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 121, in _autoSetNWorkUnits
    optimalNWorkUnits = computeOptimalNWorkUnits()
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/testWorkUnits.py", line 46, in computeOptimalNWorkUnits
    source.propagate(scene, logger=logger, showProgress=False)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 35, in propagate
    IPP = self._getAverageInteractionsPerPhoton(scene)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 62, in _getAverageInteractionsPerPhoton
    self._measureIPP(scene)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 79, in _measureIPP
    self._propagateOpenCL(estimatedIPP, scene, tempLogger, showProgress=False)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 99, in _propagateOpenCL
    self._photons.propagate(IPP=IPP, verbose=showProgress)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLPhotons.py", line 60, in propagate
    program.launchKernel(kernelName="propagate", N=np.int32(params.workItemAmount),
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLProgram.py", line 30, in launchKernel
    self._build(CLObjects)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLProgram.py", line 59, in _build
    self._program = cl.Program(self._context, sourceCode).build()
  File "/Users/marc-andrevigneault/.virtualenvs/PyTissueOptics/lib/python3.8/site-packages/pyopencl/__init__.py", line 536, in build
    self._prg, was_cached = self._build_and_catch_errors(
  File "/Users/marc-andrevigneault/.virtualenvs/PyTissueOptics/lib/python3.8/site-packages/pyopencl/__init__.py", line 584, in _build_and_catch_errors
    raise err
pyopencl._cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i7-1060NG7 CPU @ 1.20GHz' on 'Apple' at 0xffffffff>:

<program source>:197:27: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    normalizeVectorGlobal(&vectors[id]);
                          ^~~~~~~~~~~~
<program source>:94:36: note: passing argument to parameter 'vector' here
void normalizeVectorGlobal(float3 *vector){
                                   ^
<program source>:202:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&vector[i], &axis[i], angle[i]);
                           ^~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:202:40: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&vector[i], &axis[i], angle[i]);
                                       ^~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^
<program source>:685:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].er, &photons[photonID].direction, phi);
                           ^~~~~~~~~~~~~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:685:51: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].er, &photons[photonID].direction, phi);
                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^
<program source>:686:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].direction, &photons[photonID].er, theta);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:686:58: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].direction, &photons[photonID].er, theta);
                                                         ^~~~~~~~~~~~~~~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^
<program source>:731:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector,float3 *axisVector, float theta){
                          ^
<program source>:735:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector,float3 *axisVector, float theta){

(options: -I /Users/marc-andrevigneault/.virtualenvs/PyTissueOptics/lib/python3.8/site-packages/pyopencl/cl)
(source saved as /var/folders/95/r0fv6k116d1_zz6pnxh4h3lc0000gn/T/tmp90_b3v1u.cl)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/example.py", line 26, in <module>
    source = PencilPointSource(position=Vector(0, 0, 0), direction=Vector(0, 0, 1), N=N, useHardwareAcceleration=True)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 206, in __init__
    super().__init__(position=position, direction=direction, diameter=0, N=N,
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 169, in __init__
    super().__init__(position=position, N=N, useHardwareAcceleration=useHardwareAcceleration)
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/source.py", line 26, in __init__
    useHardwareAcceleration = validateOpenCL()
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/__init__.py", line 15, in validateOpenCL
    CONFIG.validate()
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 56, in validate
    self._autoSetNWorkUnits()
  File "/Users/marc-andrevigneault/Documents/Github/PyTissueOptics/pytissueoptics/rayscattering/opencl/CLConfig.py", line 127, in _autoSetNWorkUnits
    raise ValueError(f"The automatic test for optimal N_WORK_UNITS failed. Please manually run the test "
ValueError: The automatic test for optimal N_WORK_UNITS failed. Please manually run the test script 'testWorkUnits.py' and set N_WORK_UNITS in the config file at 'rayscattering/opencl/config.json'. 
... Error message: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i7-1060NG7 CPU @ 1.20GHz' on 'Apple' at 0xffffffff>:

<program source>:197:27: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    normalizeVectorGlobal(&vectors[id]);
                          ^~~~~~~~~~~~
<program source>:94:36: note: passing argument to parameter 'vector' here
void normalizeVectorGlobal(float3 *vector){
                                   ^
<program source>:202:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&vector[i], &axis[i], angle[i]);
                           ^~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:202:40: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&vector[i], &axis[i], angle[i]);
                                       ^~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^

<program source>:685:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].er, &photons[photonID].direction, phi);
                           ^~~~~~~~~~~~~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:685:51: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].er, &photons[photonID].direction, phi);
                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^
<program source>:686:28: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].direction, &photons[photonID].er, theta);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:103:38: note: passing argument to parameter 'mainVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                     ^
<program source>:686:58: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAroundAxisGlobal(&photons[photonID].direction, &photons[photonID].er, theta);
                                                         ^~~~~~~~~~~~~~~~~~~~~
<program source>:103:59: note: passing argument to parameter 'axisVector' here
void rotateAroundAxisGlobal( float3 *mainVector,  float3 *axisVector, float theta){
                                                          ^

<program source>:731:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector,float3 *axisVector, float theta){
                          ^
<program source>:735:18: error: passing '__global __float3 *' to parameter of type '__float3 *' changes address space of pointer
    rotateAround(&photons[photonID].direction, &fresnelIntersection->incidencePlane, fresnelIntersection->angleDeflection);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:153:27: note: passing argument to parameter 'mainVector' here
void rotateAround(float3 *mainVector,float3 *axisVector, float theta){
                          ^

(options: -I /Users/marc-andrevigneault/.virtualenvs/PyTissueOptics/lib/python3.8/site-packages/pyopencl/cl)
(source saved as /var/folders/95/r0fv6k116d1_zz6pnxh4h3lc0000gn/T/tmp90_b3v1u.cl)

Process finished with exit code 1

PyMarc2 commented 1 year ago

accidental close

PyMarc2 commented 1 year ago

now runs on mac os Montery 12.4. @dccote does it work for you?

DCC-Lab / PyTissueOptics