Invalid MEX-file error with fresh install

[gzmk]

Hi there,

I am trying to revive some RenderToolbox3 code and want to first make it run on RenderToolbox4 and eventually migrate it. I installed Version 4 but get an error in trying to render an example scene (in this case rtbMakeLightFieldSphere.m)

rtbMakeSimpleSphere

MakeSceneFiles started at 24-Jul-2019 15:50:26.

Using strategy RtbAssimpStrategy

strategy = 

  RtbAssimpStrategy with properties:

                                hints: [1×1 struct]
                           importArgs: {'ignoreRootTransform'  [1]  'flipUVs'  [1]}
                         mappingsArgs: {[1×1 struct]}
         remodelOnceBeforeAllFunction: []
    remodelPerConditionBeforeFunction: []
     remodelPerConditionAfterFunction: []
                            converter: [1×1 RtbAssimpMitsubaConverter]
                             renderer: [1×1 RtbMitsubaRenderer]

MakeSceneFiles started with isParallel=0 at 24-Jul-2019 15:50:27.

nConditions =

     1

conditionValues =

  0×0 empty cell array

MakeSceneFiles finished at 24-Jul-2019 15:50:27 (0.6s elapsed).

Using strategy RtbAssimpStrategy

BatchRender started with isParallel=0 at 24-Jul-2019 15:50:28.

Starting scene 1 of 1 at 24-Jul-2019 15:50:28 (0.0s elapsed).

docker run -ti --rm --user="1158290076":"1158290076" --workdir="/Users/gizem/Documents/MATLAB/render_toolbox/rtbMakeSimpleSphere" --volume="/Users/gizem/Documents/MATLAB/render_toolbox/rtbMakeSimpleSphere":"/Users/gizem/Documents/MATLAB/render_toolbox/rtbMakeSimpleSphere" --volume="/Users/gizem/Documents/MATLAB/toolboxes/RenderToolbox4":"/Users/gizem/Documents/MATLAB/toolboxes/RenderToolbox4" rendertoolbox/mitsuba-spectral mitsuba -o "/Users/gizem/Documents/MATLAB/render_toolbox/rtbMakeSimpleSphere/renderings/Mitsuba/scene-001.exr" "/Users/gizem/Documents/MATLAB/render_toolbox/rtbMakeSimpleSphere/scenes/Mitsuba/scene-001.xml"
Invalid MEX-file '/Users/gizem/Documents/MATLAB/toolboxes/RenderToolbox4_binary/build/ReadMultichannelEXR.mexmaci64':
dlopen(/Users/gizem/Documents/MATLAB/toolboxes/RenderToolbox4_binary/build/ReadMultichannelEXR.mexmaci64, 6): Library
not loaded: /usr/local/lib/libIlmImf-2_2.22.dylib
  Referenced from: /Users/gizem/Documents/MATLAB/toolboxes/RenderToolbox4_binary/build/ReadMultichannelEXR.mexmaci64
  Reason: image not found.

Error in rtbReadMultichannelEXR (line 23)
[channelInfo, imageData] = ReadMultichannelEXR(exrFile);

Error in rtbReadMultispectralEXR (line 39)
[sliceInfo, imageData] = rtbReadMultichannelEXR(exrFile);

Error in RtbMitsubaRenderer/render (line 124)
            [image, ~, sampling] = rtbReadMultispectralEXR(outFile);

Error in rtbBatchRender>renderScene (line 101)
    strategy.renderer.render(scene);

Error in rtbBatchRender (line 85)
        outFiles{ii} = renderScene(strategy, nativeScenes{ii}, versionInfo, hints);

Error in rtbMakeSimpleSphere (line 34)
    radianceDataFiles = rtbBatchRender(nativeSceneFiles, ...

My system is Mac OS 10.14, MATLAB_R2018a. I tried to downgrade openexr thinking it is an unsupported version issue but couldn't get that to run either. I even tried converting back to RenderToolbox3 with no success. Also, I was able to run rtbMakeReadMultiChannelEXR.m and generate the output sphere images so at this point I am not even sure if the error is due to OpenEXR or something else.

Thanks for the help! Gizem

[DavidBrainard]

Yikes - I got no email pointing me to this. It was just pointed out to me by someone else having the same issue.

We will look into it.

[DavidBrainard]

See issue 25 for a possible fix, although given that you have recompiled the EXR read mex file I am not sure you are dealing with the same issue. Let us know and we'll try to help.

[jennybosten]

Hi, I seem to have a similar issue. When attempting to run rtbTestInstallation(); it works down to the point where I get errors such as 4 rtbMakeMaterialSphereRemodeled.m MException with properties:

identifier: 'MATLAB:mex:ErrInvalidMEXFile'
   message: 'Invalid MEX-file '/Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64': dlopen(/Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64, 6): Library not loaded: /usr/local//libassimp.3.dylib↵  Referenced from: /Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64↵  Reason: image not found'
     cause: {}
     stack: [8×1 struct]
Correction: []

Invalid MEX-file '/Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64': dlopen(/Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64, 6): Library not loaded: /usr/local//libassimp.3.dylib Referenced from: /Users/jb633/Documents/MATLAB/toolboxes/mexximp_binary/build/mexximpConstants.mexmaci64 Reason: image not found

I can't find issue 25.

Also I can run rtbMakeReadMultiChannelEXR.m and make the output sphere. I am running Catalina (10.15.7) and Matlab 2020b. All the dependencies seem to be installed OK.

[DavidBrainard]

I am not sure where issue 25 went. We did recently succeed in doing fresh installs on the Mac, which required rolling back to an older version of assimp. The wiki install instructions were updated with that info. What platform are you installing on?

[jennybosten]

Ah, thanks, I had been using a text file in the toolbox for the install instructions... I am on MacOS 10.15.7. I decided to do a clean install with the instructions on the wiki. I have run into some errors installing Assimp with the instructions on the wiki, but there is a success message at the end. I can't see it in usr/local/Cellar though (the version 5 I had in there before seems to have uninstalled). Here was the output:

fatal: --unshallow on a complete repository does not make sense Homebrew repo already unshallowed Uninstalling assimp... (100 files, 12.3MB) Error: Calling Non-checksummed download of assimp formula file from an arbitrary URL is disabled! Use 'brew extract' or 'brew create' and 'brew tap-new' to create a formula file in a tap on GitHub instead. assimp 4.1.0 installed.

I also need to request permission to access https://github.com/imageval/iset.git ?

[DavidBrainard]

You don't really need iset to use RTB4. But in any case it has now moved to https://github.com/iset/isetcam.git. I have updated the instructions.

[DavidBrainard]

Here's what I get if I type "brew info assimp" at the terminal.

dhb@Davids-iMac Documents % brew info assimp assimp: stable 5.0.1 (bottled), HEAD Portable library for importing many well-known 3D model formats https://www.assimp.org/ /usr/local/Cellar/assimp/4.1.0 (61 files, 7.7MB) * Poured from bottle on 2020-08-04 at 17:54:09 From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/assimp.rb License: LGPL-3.0 ==> Dependencies Build: cmake ✘ ==> Options --HEAD Install HEAD version ==> Analytics install: 1,389 (30 days), 3,903 (90 days), 16,098 (365 days) install-on-request: 494 (30 days), 1,413 (90 days), 6,976 (365 days) build-error: 0 (30 days) dhb@Davids-iMac Documents % ls /usr/local/Cellar assimp gradle jpeg openexr ghostscript ilmbase libtiff openjdk dhb@Davids-iMac Documents %

Looks like your install didn't work. I will see if I can do this again on my machine.

[DavidBrainard]

I fixed the README to refer to the wiki instructions for installation.

[DavidBrainard]

I now get the same error you do with brewv. Something has changed since August in the way brew works and/or checks things. We will have to come up with an alternate method of getting the old version of assimp. Working on it.

[jennybosten]

Thanks! I am currently attempting to install that version of Assimp from the source code... will see how that goes.

[DavidBrainard]

Let me know if you get a solution. My understanding is that is basically what brewv is trying to do - invoke brew itself in some way that goes to the right git version. I've asked the expert in my lab (Nicolas Cottaris) to take a look, but if you find the right magic incantations first let me know.

[jennybosten]

Thanks David for your help. After installing CMake and then using it to install Assimp 4.1.0 and altering some paths in makeMexximp.m to point to where I installed Assimp, the mex file seemed to create successfully but (update) I still have the mex file error. I will give it another go tomorrow with a fresh brain!

Further update: I had renamed the Assimp folder which created some kind of conflict so I named it back, re-ran makeMexximp.m and then reset all the paths. rtbMakeChessSet then ran a long way before it ran into an error

Unrecognized function or variable 'hints'. Error in rtbMakeChessSet (line 146) 'hints', hints);

It ran up to that line though and I got a chess set from line 138 :). I think the problem is ieInit clears the workspace.

[DavidBrainard]

Does rtbTestInstallation work OK? rtbMakeChessSet runs fine on my machine, but I don't have isetcam on my path and thus don't have ieInit. I think your guess is right. If you move the ieInit call to the start of the script inside a conditional, and then have the same conditional at the end but without the ieInit, that might fix it. If you don't plan to use isetcam or isetbio with RTB, you can just take the isetcam off your path and that should also fix this.

[jennybosten]

rtbTestInstallation produces an error:

MException with properties:

identifier: 'MATLAB:UndefinedFunction'
   message: 'Undefined function 'MMitsubaElement' for input arguments of type 'char'.'
     cause: {}
     stack: [9×1 struct]
Correction: []

Undefined function 'MMitsubaElement' for input arguments of type 'char'.

Perhaps there are still elements that aren't correctly installed.

[DavidBrainard]

I'll have to dig a little more to figure this out. May not get to it until weekend. Is it easy to post here instructions on how you used CMake to get and use the older assimp version?

[jennybosten]

Sure, I downloaded cmake and used these line in readme.rst $ ./bootstrap && make && sudo make install

to build and install. I then downloaded the source code for Assimp 4.1.0 (https://github.com/assimp/assimp/releases/tag/v4.1.0/) and followed the INSTALL text file running

cmake CMakeLists.txt -G 'Unix Makefiles'

then

make

Assimp ended up installed in a location other than that expected in makeMexximp.m so I changed some of the paths in that script. Hopefully it doesn't cause further problems down the line.

[DavidBrainard]

Thanks. I'll get back to you as soon as I can on the MException above.

[DavidBrainard]

a) Nicolas has updated brewv.sh so that it works again. Thanks Nicolas. b) I'll add your instructions above to the wiki as a backup plan. c) rtbTestInstallation didn't work for me on the first pass because I needed to remake the ReadMultichannelEXR mex file. Not sure why, but in any case once I did that and made sure the new version was the one that was first on Matlab's path, it then ran without error. d) If that doesn't fix it for you, can you send me the full error dump from Matlab?

[jennybosten]

I remade the ReadMultichannelEXR mex file but still got the same error - here is all the stuff spat out when running rbTestInstallation:

Checking working folder: folder exists: /Users/jb633/Documents/MATLAB/render_toolbox OK. Trying to write: /Users/jb633/Documents/MATLAB/render_toolbox/test.txt OK. Checking for OpenEXR: brew list | grep openexr OK. Checking for Docker: docker ps OK.

Testing rendering with 4 example scripts. You should see several figures with rendered images.

MakeSceneFiles started at 13-Nov-2020 09:26:23.

MakeSceneFiles started at 13-Nov-2020 09:26:23.

MakeSceneFiles started at 13-Nov-2020 09:26:23.

0 scenes succeeded.

4 scenes failed.

---

1 rtbMakeCoordinatesTest.m MException with properties:

identifier: 'MATLAB:UndefinedFunction'
   message: 'Undefined function 'MMitsubaElement' for input arguments of type 'char'.'
     cause: {}
     stack: [11×1 struct]
Correction: []

Undefined function 'MMitsubaElement' for input arguments of type 'char'.

---

2 rtbMakeDragon.m MException with properties:

identifier: 'MATLAB:UndefinedFunction'
   message: 'Undefined function 'MMitsubaElement' for input arguments of type 'char'.'
     cause: {}
     stack: [11×1 struct]
Correction: []

Undefined function 'MMitsubaElement' for input arguments of type 'char'.

---

3 rtbMakeMaterialSphereBumps.m MException with properties:

identifier: 'MATLAB:UndefinedFunction'
   message: 'Undefined function 'MMitsubaElement' for input arguments of type 'char'.'
     cause: {}
     stack: [11×1 struct]
Correction: []

Undefined function 'MMitsubaElement' for input arguments of type 'char'.

---

4 rtbMakeMaterialSphereRemodeled.m MException with properties:

identifier: 'MATLAB:UndefinedFunction'
   message: 'Undefined function 'MMitsubaElement' for input arguments of type 'char'.'
     cause: {}
     stack: [9×1 struct]
Correction: []

Undefined function 'MMitsubaElement' for input arguments of type 'char'.

Elapsed time is 1.023821 seconds.

No referenceRoot provided. Local renderings will not be compared with reference renderings.

Perhaps I should re-install Assimp using brev.sh? I don't seem to have a file named MMistubaElement but I wondered whether the problem might lie with mMitsubaImportMexximp?

[DavidBrainard]

I just did a which on mMitsubaElement - not sure why I didn't think to do that before.

which MMitsubaElement /Users/dhb/Documents/MATLAB/toolboxes/mMitsuba/api/MMitsubaElement.m % MMitsubaElement constructor

It's in its own toolbox: "url": "https://github.com/RenderToolbox/mMitsuba.git", so you'll need that.

I looked more carefully at our ToolboxToolbox configuration for RenderToolbox4. Looks we also put on our path

"url": "https://github.com/RenderToolbox/mPbrt.git" "https://www.mathworks.com/matlabcentral/mlc-downloads/downloads/submissions/59411/versions/1/download/zip" "https://www.mathworks.com/matlabcentral/mlc-downloads/downloads/submissions/28518/versions/9/download/zip"

"url": "https:\/\/github.com\/RenderToolbox\/pbrt2iset.git" "https://github.com/RenderToolbox/SphereRendererToolbox.git" "url": "https://github.com/Psychtoolbox-3/Psychtoolbox-3.git"

I think you really will need the first three. I think the second three are more optional.

Sorry that the documentation got stale.

We do all our configuration using ToolboxToolbox (https://github.com/toolboxhub/ToolboxToolbox.git), and if you have that set up, you can get the whole ball of wax using the single comment tbUse('RenderToolbox4') at the Matlab prompt.

[jennybosten]

It worked! :) I had used the ToolboxToolbox installation on an earlier attempt but decided to get rid of everything and start afresh from the wiki page. Actually it was probably all fine apart from Assimp. I can't find the instructions for the tbUse route any more though?

Thanks David for all the time you have spent helping me install this. A new PhD student in the lab is going to be using it for her colour constancy work.

[DavidBrainard]

Super. Happy to help, sorry it was so painful. Let me know if you get stuck with anything else going forward.

The instructions for installing ToolboxToolbox are at https://github.com/toolboxhub/ToolboxToolbox.git.

Then for RenderToolbox4 it's just "tbUse('RenderToolbox4')"

Best,

David

From: jennybosten notifications@github.com Reply-To: RenderToolbox/RenderToolbox4 reply@reply.github.com Date: Friday, November 13, 2020 at 9:30 AM To: RenderToolbox/RenderToolbox4 RenderToolbox4@noreply.github.com Cc: "Brainard, David H" brainard@psych.upenn.edu, Comment comment@noreply.github.com Subject: Re: [RenderToolbox/RenderToolbox4] Invalid MEX-file error with fresh install (#42)

It worked! :) I had used the ToolboxToolbox installation on an earlier attempt but decided to get rid of everything and start afresh from the wiki page. Actually it was probably all fine apart from Assimp. I can't find the instructions for the tbUse route any more though?

Thanks David for all the time you have spent helping me install this. A new PhD student in the lab is going to be using it for her colour constancy work.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/RenderToolbox/RenderToolbox4/issues/42#issuecomment-726795552, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAI24ZKNRZONIKCDNF4RXMTSPU7GTANCNFSM4IGVPUMA.

[DavidBrainard]

Updated documentation and closing.