problems when running the spaintgui.exe

[cy19971107]

I think I build the spaint successfully, but when I run the spaintgui.exe I get the result as following:

[spaint] Found 0 joysticks. Cannot get the RGB image size from an empty composite image source engine

What's the reason of getting this result? and when I run this with openNI and want to open the picture in InfiniTAM: ./spaintgui Teddy/calib.txt Teddy/Frames/%04i.ppm Teddy/Frames/%04i.pgm I get: [spaint] Found 0 joysticks. [spaint] Probing OpenNI camera: Default Calibration filename not specified. Using default parameters. compiled without OpenNI support Cannot get the RGB image size from an empty composite image source engine

Why I cannot open the file with spaint?

Thank you

[sgolodetz]

It looks like you've built without OpenNI support - if you rebuild with that enabled (for both InfiniTAM and spaint) and plug in a supported camera then it should work.

Regarding the one with the sequence, I think I need to update the README - the way in which you specify sequences on the command line got upgraded, but the README wasn't updated accordingly. The way it works now is that you can specify the calibration file, depth mask and RGB mask separately via '-c', '-d' and '-r'.

Edit: The updates to the README are here (3db43e594e87e511ba4257eccdc6010ad0ef02e6) - I'm just in the process of checking them over, making any additional changes and getting them merged.

[cy19971107]

I run the spaintgui.exe successfully and open the file. But I have another question: "plug in a supported camera" what kind of camera is supported? only Kinect? thank you.

[sgolodetz]

Kinect, PrimeSense, Asus Xtion, Orbbec Astra, Intel RealSense, Zed. Some of those will need additional libraries/drivers installing though. The easiest is to use one that's supported by OpenNI, like a Kinect or Xtion.

[cy19971107]

Only have Intel Creative Senz3D to use……Is it supported? And if I want to use additional libraries/drivers ( for example, Creative Senz3D camera would work with Intel Perceptual Computing (PerC) SDK ,I have installed it),how can I connect it to the spaint project?

[sgolodetz]

You'll probably need to write your own ImageSourceEngine - see the InputSource project in InfiniTAM.

[cy19971107]

What version of Kinect can be used for this project?1.0 or 2.0?

[sgolodetz]

Either in principle, in the sense that I think @victorprad had a Kinect 2 image source engine implemented in InfiniTAM. I've only used the Kinect 1 one though, as I don't personally have a Kinect 2. From what I remember, the Kinect 2 didn't work so well - so not sure what the current state of the image source engine is for that. Probably wouldn't be too hard to get it up and running though.

[cy19971107]

It seems that Kinect 1 is better, I will try to get it. Thank you

[cy19971107]

I have connected the camera and get picture I want successfully, but now I have some trouble interacting with scene.It seems that I need to choose WITH_ARRAYFIRE WITH_CUDA WITH_OPENMP WITH_OPENNI Is WITH_CUDA must be choosen for interacting with scene ?

[sgolodetz]

You need ArrayFire for touch interaction. Can't remember if it can work without CUDA - wouldn't recommend it if you can avoid it, will be slow.

[cy19971107]

sorry to bother you but I find an important bug when buiding the spaintgui. I search the internet,and I guess the reason is code error but I don't know how to deal with it. the bug is : spaint_d.lib(spaint_intermediate_link.obj) : error LNK2005: __cudaRegisterLinkedBinary_30_ztrmm_sm35_cpp2asm_asm_cpp1_ii_2af62f32 already defined in spaintgui_intermediate_link.obj …… itmx_d.lib(itmx_intermediate_link.obj) : error LNK2005: __cudaRegisterLinkedBinary_30_ztrmm_sm35_cpp2asm_asm_cpp1_ii_2af62f32 already defined in spaintgui_intermediate_link.obj …… fatal error LNK1169: one or more multiply defined symbols found

[sgolodetz]

@cy19971107: Try rerunning the build script from Git Bash - I find I get these errors sometimes also, I think they're caused by stale files. Rerunning the build script seems to fix it.

[cy19971107]

I try a whole day but the method you provided didn't work.The bug always occurs. My system is WIN10 64bits, using VS2013 to build to project. I search the bug:error LNK2005 on the Internet: when a header file defines a variable. For example, if you include this header file in more than one source file in your project, an error results. In the spaint I find the CUDADefines.h is used by spaint, itmx and spaintgui, I guess maybe this is the reason of the bug.What should I do? Maybe this bug only occurs on Windows, but why Other systems don't have this bug? Thank you.

[sgolodetz]

Hmm, CUDADefines.h doesn't define any variables obviously. Why do you think it's that? Can you email me your CMakeCache.txt file from the build directory? Might help me have a think about what's going on.

[sgolodetz]

This might be relevant:

https://gitlab.kitware.com/cmake/cmake/issues/17520

[cy19971107]

@sgolodetz CMakeCache.txt

Here is my CMakeCache.txt

[cy19971107]

Some bugs occur when I use boost 1.56.0 to compile so I use the boost 1.67.0. The next two pictures is the build error in VS2013.

[sgolodetz]

So one thing that occurred to me was that it might be a CMake version issue - they added first-class support for CUDA recently, and I found on Mac that my build broke when I tried to use CMake 3.11 rather than 3.9. You could try an older version of CMake to see if that fixes it - it might just be that I need to upgrade the build to work with new CMake. I'm building with 3.9 or older on all platforms, and it works happily enough on a variety of different machines, so that's a likely suspect for why it might not be working for you.

[sgolodetz]

What build errors were you having with Boost 1.56.0?

[sgolodetz]

One other thing that looks weird in your CMakeCache.txt is the location of InfiniTAM. Are you building against the infinitam_v3.5 branch? The master branch of spaint is set up to build against that, by default looking in a sibling directory of spaint called infinitam (lower case).

[sgolodetz]

You may have older versions of both InfiniTAM and spaint fetched also - if so, might be a good idea to update as the folder structure changed in InfiniTAM from 3 -> 3.5 (we changed InfiniTAM/InfiniTAM to infinitam).

[cy19971107]

build errors with Boost 1.56.0： error C2664: 'void std::_Tree<std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,false>>::insert(std::initializer_list<std::pair<const _Kty,_Ty>>)' : cannot convert argument 1 from 'std::pair<const void *,boost::shared_ptr<const rafl::Example>>' to 'std::pair<const _Kty,_Ty> &&'……

[sgolodetz]

Did you build Boost with the build script?

https://github.com/torrvision/spaint/blob/master/libraries/build-boost_1_56_0-win.sh

There's a fix for shared_ptr_helper.hpp embedded in that. If you built manually, you might not have picked it up.

[cy19971107]

Now I build the spaint successfully! I think boost 1.67.0 is still ok for this project. I think the bug I had yesterday is caused by cmake 3.11.0, now I use cmake 3.9 so the bug does not occur. But it seems that I need afcuda.dll for running the spaintgui.exe , where can I find this file?

[sgolodetz]

Yes, Boost 1.67.0 should work too - don't think there are any serious interface changes that will affect it. I should update the readme to say CMake 3.9 for now - apparently 3.11 needs some changes to work. The missing DLL is in the ArrayFire bin directory IIRC - the build should copy it across by default, but you can always do it manually if not.

[sgolodetz]

Hmm maybe it doesn't at the moment for some reason:

https://github.com/torrvision/spaint/blob/master/cmake/LinkArrayFire.cmake

You might also need the nvvm DLL for CUDA 9. I've been using an earlier CUDA version, so the LinkArrayFire.cmake script may need updating slightly for CUDA 9.

[cy19971107]

OK I copy the dll file which ArrayFire needs by my hand, and the spaintgui.exe can run successfully. I still have some problems. 1.Do grove need to be built if I want to interacting with scene ? 2.My build options are here, What else do I need to choose?

3.I want to try how to color the image I get in the spaint,I enter I +4 to spaint ,but I don't know what to do next. 4.The spaint is running too slowly,how can I make it faster?

[sgolodetz]

1. Only if you want the Grove relocaliser. It's better than ferns, particularly for novel poses, but ferns is simpler, and ok if you're not too worried about relocalisation quality per se.
2. What problems are you experiencing?
3. I+2 is mouse labelling, I+4 is touch interaction. If you want to use touch interaction, you point the camera at the scene and touch the scene in the real world. You should turn off fusion first though (with the F key). Also, you need to make sure your arm is far enough away from the camera (so there's depth for it), and you need to make sure that your arm is never taking up most of the image (you don't want the tracker to track against the arm rather than the background).
4. What's your machine spec? Did you build both InfiniTAM and SemanticPaint in Release mode? And please can you pin down "too slowly"? How slow is it for you?

p.s. See https://gitlab.kitware.com/cmake/cmake/issues/17987 for what looks like the same CMake 3.11 issue we were having.

[sgolodetz]

Btw, just as a side note, if you want to build Grove ever, you'll need to enable ALGLIB as well.

[cy19971107]

I build both InfiniTAM and SemanticPaint in Release mode. I enter the I+4 and touch the scene in the real world ，but nothing happened. Also, After I enter I+4 to the program,after a shourt time the spaintgui.exe will close by itself automatically although I don't want it to close.

[sgolodetz]

What happens if you press I+2 and then label the scene with the mouse? Does that work?

If you run spaintgui from the command line and press I+4, do you get any error message in the terminal (e.g. an exception message)?

[sgolodetz]

Does scratchtest_arrayfire run ok?

[sgolodetz]

What GPU are you using? (Just wondering if you're running out of GPU memory or something.)

[cy19971107]

I press I+2 and there is a orange point on my mouse , what should I do to label the scene?

[sgolodetz]

Click the scene. Also, press H for the shortcut keys to change label / labelling size.

[cy19971107]

The result of running scratchtest_arrayfire

I think there is no problem with arrayfire……

The thing I want to do is touch the object in the real scene, use voice control to color the thing I touch.Just as the video shows on http://mmcheng.net/zh/s-paint/……

[sgolodetz]

For the voice, you need to use a separate Java application called voicecommander - I'm about to drive home, so I'll have to help with that later. The GPU on your machine looks a bit underpowered I think, which might explain some of the problems you're having:

https://www.notebookcheck.net/NVIDIA-GeForce-940MX.156033.0.html

Will have to try to debug what's going on later.

Bear in mind that it won't look exactly like Ming's video, since there are two different implementations and that's from the other one, but if we get it working it will be close.

[cy19971107]

Ok, I hope you can help me with these problems when you have time. 1.So I need to install the voicecommander?where can I get this application,how can I connect it to spaint? 2.“The GPU on your machine looks a bit underpowered” But this is not the thing I can control……Can I finish the project on this GPU?

[sgolodetz]

1. You can clone it from here: git@bitbucket.org:sgolodetz/voicecommander.git

When you run it, it runs a local voice command server to which spaintgui will connect (start voicecommander first).

2. Not sure, will need some debugging (and possibly decreasing the number of locally stored blocks in ITMVoxelBlockHash.h, if it's a memory issue). As a starting point, not sure if you saw my question above - if you run spaintgui from the command line and press I+4, do you get any error message in the terminal (e.g. an exception message)?

[sgolodetz]

Re. GPU performance btw - for reference, I have a GeForce 560M on my Macbook, and this is how your GPU stacks up against it:

http://www.game-debate.com/gpu/index.php?gid=504&gid2=3455&compare=Nvidia%20GeForce%20GTX%20560M-vs-Nvidia%20GeForce%20940MX%202GB

SemanticPaint really grinds on my Macbook (vs. the sort of laptop we use for a conference demo, which has a 980M in it):

http://www.game-debate.com/gpu/index.php?gid=3455&gid2=2443&compare=Nvidia%20GeForce%20940MX%202GB-vs-Nvidia%20GeForce%20GTX%20980M%208GB

So I think what I'm trying to say is - no promises on how well it will work on your GPU unfortunately. It hasn't been optimised for that kind of GPU, and with only 2GB of memory, you're quite likely to hit problems. I had to manually decrease the memory usage to get things to run ok on my Macbook. You could try turning on USE_LOW_POWER_MODE in CMake - that turns off colour information in the voxels, which will save some memory.

[cy19971107]

I didn't get any error message in the terminal from the command line when I enter I +4 to spaint, it just close automatically. BUt there is a bug when I run the RUN_TEST in vs2013: now I build the voicecommander successfully, what should I do next?

[sgolodetz]

So you should be able to do ant run to run it, and then spaintgui should be able to connect to it. If you plug in a microphone, then say different commands into it, their effects should happen in spaintgui. See in particular Application::process_voice_input() for the commands you can use.

Regarding the test failure - try putting a breakpoint at the start of copy_image_test and stepping through to see where it fails. The two problems are very likely related, since it's probably something to do with the use of ArrayFire. If it doesn't get as far as copy_image_test, try putting a breakpoint in the CopyImageFixture constructor instead.

[cy19971107]

I open it and connect it to spanit . BUT how can I close the voicecommander ?I said 'quit' but nothing happended.

[sgolodetz]

Ctrl + C? There's no "quit" voice command as things stand - but see Application::process_voice_input() if you want to add one.

[cy19971107]

I enter Ctrl + C but the voicecommander did not close, I have to close the git bash to make it close. Now I run the RUN_TEST and all the tests pass. I think the bug before is caused by some dll files loss.I add them so all the tests pass. But my spaint still can not run well.I enter I+4 and get this:

[cy19971107]

@sgolodetz
Oh my god I find an amzing thing. I try not to build InfiniTAM and spaint with CUDA. Then I enter I+4 to spaintgui.exe and I run the program successfully. I think maybe CUDA is not so useful on my computer:) Thank you for all your help these days.

[sgolodetz]

No problem - glad it works ok in CPU mode at least!