psmove_tracker_new hangs if called more than once

[DrummerB]

Hi,

I'm trying to get psmove to work in Unity, including tracking. I've looked at UniMove and UniMoveX and was able to get it all up and running on a Mac with the included PSEye camera.

However, there seems to be an issue in psmove_tracker_new that causes a freeze, when called more than once. When I first start the game within the editor, everything works fine. If I then stop and restart the game, the entire editor hangs and crashes. I tracked the crash to the initial call of psmove_tracker_new.

It's easy to reproduce this issue without Unity or UniMoveX. For instance, you can modify the test_tracker program to just do the whole thing twice. It crashes on the second run:

https://gist.github.com/DrummerB/76bdfd7cc2a106f5a367

[rovarma]

Thanks for reminding me of this.

This is a bug I introduced with https://github.com/thp/psmoveapi/pull/200; it was fixed by @brendanwalker a while ago in https://github.com/inspirit/PS3EYEDriver/commit/73c8651841f6017d9d79218ed35300ce2b2c2631, but I completely forgot to update the PS3EYEDriver submodule reference for psmoveapi to point to that commit, so it still has the problem.

I'll open a PR to update the PS3EYEDriver reference tonight, but in the meantime, you can apply his patch locally to work around this (as you can see from the commit, it's a very small change).

[DrummerB]

Thank you so much!

I applied that change to psmoveapi (it's just one line in the PS3EYEDriver, right?) and rebuild to dylibs. The simple test case I included in the report above works fine now. The tracker seems to correctly initialize and destroy multiple times in a row.

In Unity and UniMoveX it still hangs on the second launch of the game inside the editor, eventhough I'm using the same dylibs. I'm trying to build a simple test case now to see if the issue is with UniMoveX or psmoveapi.

Thanks again :+1:

[rovarma]

Yes, it's only that single line (setting the bool to false). I'm unfamiliar with UniMoveX, but if you can post some callstacks of when it's hanging, I could at least have a look to see if anything looks suspicious.

[cboulay]

Instead of UniMove, check out psmove-unity5

[DrummerB]

It looks like it's an issue with UniMoveX. I was able to recreate the test_tracker program within Unity and I'm able to restart the game multiple times without a crash and getting tracking info. I'm not sure whether I should try to fix UniMoveX or just roll my own. I'll update when I make any progress. Thanks!

[DrummerB]

@cboulay I've seen that project, but it seems to be Windows only. Since you seem to be an active participant on that project, do you have a guess how difficult it would be to use it on OS X? The DK2 coregistration and reduced drift certainly sound interesting.

[cboulay]

Actually I've never used Unity, UniMove or psmove-unity5. I was more involved in psmove-ue4, which psmove-unity5 is based on. The main thing they share in common is that they both use my fork of psmoveapi.

Looking at psmove-unity5, I don't see anything platform specific other than the wiki. It should work with the OSX binaries as well as the Windows binaries. But let's ping @HipsterSloth to see if he has any input.

The reduced drift is now in the upstream psmoveapi so there shouldn't be any difference there. See 4b3755368bb3161d647ed037ba894999355e8ea1

The DK2 coregistration is not, and some non-trivial changes are required to enable that (namely, how the 3D position is calculated and adding an additional transformation matrix into that process). So, if the DK2 coregistration is important to you, use my psmoveapi fork, for now.

[DrummerB]

I see, thanks for clarifying. I will take a deeper look at psmove-unity5.

[HipsterSloth]

Oh sorry, psmove-unity5 can definately be made to work on OS X. I've just been too lazy to get around to compiling my fork of psmoveapi that I use for psmove-unity5 on OS X. I can do that tonight and check in the binaries.

[DrummerB]

I would really appreciate an OSX build, thank you @HipsterSloth!

Wouldn't there be an issue with the out-of-date Oculus SDK for OS X? The latest SDK available is 0.5. I just tried compiling the master branch of @cboulay's fork and I get the following message when running cmake:

Could NOT find OCULUS_SDK (missing: OCULUS_SDK_LIBRARIES OCULUS_SDK_INCLUDE_DIRS)

I have 0.5 installed, which works ok with Unity, but maybe psmoveapi expects a more recent version? I see that there is at least one commit mentioning 0.8.

Upon running make I then get the following error, which is probably unrelated.

[ 92%] Building C object CMakeFiles/tracker_camera_calibration.dir/examples/c/tracker_camera_calibration.c.o
/Users/Bazsi/Coding/Libraries/psmoveapidk2/examples/c/tracker_camera_calibration.c:45:14: fatal error: 'opencv2/calib3d/calib3d_c.h' file not found
    #include "opencv2/calib3d/calib3d_c.h"
             ^
1 error generated.

In contrast to thp's fork, cboulay's fork doesn't seem to have an external/opencv folder. Is this intended?

I don't know if I'm looking at the right place, but in the file build/CMakeFiles/tracker_camera_calibration.dir/DependInfo.cmake the following paths are mentioned as search paths:

  "/usr/local/include/opencv"
  "/usr/local/include"

I've installed opencv previously with the packet manager Homebrew and I do have a /usr/local/include/opencv folder, but calib3d is indeed missing.

$ ls /usr/local/include/opencv
cv.h        cv.hpp      cvaux.h     cvaux.hpp   cvwimage.h  cxcore.h    cxcore.hpp
cxeigen.hpp cxmisc.h    highgui.h   ml.h

[cboulay]

I know I had the Oculus SDK working with the dk2_coreg in OS X at some point using the latest official OS X ovr library(I think that's 0.5). Is your Oculus SDK installed in one of these folders?

I don't have opencv in external. I use opencv from brew. I honestly can't remember if my last build was with opencv or opencv3. If an upgraded version of opencv(2) doesn't work for you then try opencv3.

Also, you can use the pre-compiled binaries found here. I think you'll have to fix the rpath following step 7 here. Those binaries are a little old by now.

[rovarma]

@DrummerB I've committed a change to update the PS3EYEDriver submodule to the latest commit as part of https://github.com/thp/psmoveapi/pull/217; once that's accepted the issue should be fixed.

[DrummerB]

@rovarma Cool, thanks a lot!

@cboulay The OculusSDK is installed in ~/Library/Frameworks/LibOVRRT_0.framework

And I also installed opencv with brew, which installed version 2 by default. I'll try the binaries, thanks!

[cboulay]

@DrummerB you can get opencv3 with brew install opencv3.

As for OculusSDK, you can try passing the following argument to the cmake command: -DOCULUS_SDK_ROOT_DIR=~/Library/Frameworks/LibOVRRT_0.framework If that works for you then I'll add that to the list of folders in the cmake file.

[HipsterSloth]

@DrummerB I just checked in compiled binaries of psmoveapi into psmove-unity5, see here. That said, I built this on my super ancient mac mini that doesn't seem to be able to run Unity5 nor do I have a HDMI port to connect my DK2 so I have no idea if the built libraries actually work on OS X. I was able to test controller pairing and run the magnetometer calibration, but that's about all I can verify. I did the following to build my fork of psmoveapi:

1. brew install opencv
2. brew install SDL2
3. git clone --branch psmove_unity5 --recursive https://github.com/HipsterSloth/psmoveapi.git psmoveapi
4. Download https://static.oculus.com/sdk-downloads/ovr_sdk_macos_0.5.0.1.tar.gz and unzip into psmoveapi/external/OculusSDK
5. mkdir psmoveapi/build
6. cd psmoveapi/build
7. cmake -G "Xcode" ..
8. Open the generated project in the build folder in XCode
9. Delete bad "calib3d_c.h" header include in tracker_camera_calibration.c. It looks like /usr/local/include/opencv2/calib3d.hpp is a valid file to include in OS X.
10. Build the "Product"

[thp]

217 has been merged, please see if this fixes the issue.

[DrummerB]

@thp It indeed fixes the original issue, thank you!

@HipsterSloth Thank you for the binaries and the instructions! I think you hardcoded a path when building the visual_coregister_dk2 binary. When I try to run it, I get this error:

dyld: Library not loaded: /Users/<your username>/psmoveapi/build/Debug/libpsmoveapi_tracker.3.dylib

I'll try to build too and report back.

[DrummerB]

@HipsterSloth Building worked out fine. Minor details:

The correct opencv path for me was /usr/local/include/opencv2/calib3d/calib3d.hpp.

I enabled PSEye in ccmake, because I have no iSight camera, which seems to be the default in your branch. When I start test_tracker, it tries to calibrate the iSight exposure, saying that I should bringt the Move close to it and press the move button. When I do that, it crashes with an error related to the AVFoundation, probably because it can't find an iSight camera.

After enabling PSEye, setting the libusb paths and setting -std=gnu++11 I was able to compile the project and run the visual coregistration.

The Unity project is still trying to load the dll however (on OS X), resulting in this error:

EntryPointNotFoundException: LoadLibrary
PSMoveWorker.LoadLib (System.String path) (at Assets/psmove-unity5/PSMoveManager.cs:795)
PSMoveWorker.WorkerSetup (.PSMoveWorkerSettings workerSettings) (at Assets/psmove-unity5/PSMoveManager.cs:271)
PSMoveWorker.OnGameStarted (.PSMoveWorkerSettings workerSettings) (at Assets/psmove-unity5/PSMoveManager.cs:227)
PSMoveManager.Awake () (at Assets/psmove-unity5/PSMoveManager.cs:89)

Any ideas how I should load the libraries on OSX?

This is what UniMove did, but I'm not sure if this would be compatible with your approach.

[HipsterSloth]

@DrummerB Glad to hear you got my branch of psmoveapi building. I'll try and check-in binaries that enable the PSEye (sorry I forgot about that). As for the dll loading in unity it looks like I was specifying the .dll suffix on the psmove api call attributes. I think if I strip the ".dll" suffix off the attribute and rename libpsmoveapi.dylib to just psmoveapi.dylib then mono will try the appropriate file suffix on each platform. I'll try patching that up tonight.

[HipsterSloth]

@DrummerB I updated my fork of psmoveapi with the build fixes for OS X you mentioned. I rebuilt psmoveapi with the ps3eye tracker enabled (via -DPSMOVE_USE_PS3EYE_DRIVER=1) and verified that the test tracker and test opengl apps worked in OS X. The new OS X compiled libs have been added to psmove-unity5 in a new OS X branch. The libs have the same name as the dlls, but with the .dylib extension. I also removed the ".dll" suffix in the psmoveapi call attributes. I tested that the windows version of psmove-unity5 still works with these changes. Would you mind cloning the OS X branch and see if this works for you?

[DrummerB]

@HipsterSloth Thanks for your work. The binaries work fine now. Unity however doesn't seem to like .dylib files (I got the same DLL not found exception as before on the first call to the psmoveapi).

I've read that you have to bundle them in a .bundle using an Xcode "Bundle" project. In simple cases simply changing the extension also works. This is what I've tried and the initial error was solved. Unity now seems to be able to correctly load the library. However there seems to be a hardcoded reference to psmoveapi_tracker inside the base psmoveapi library. I get an error about a missing library at /Users/<your username>/psmoveapi/.... Or maybe I'm doing something wrong.

I've not yet been able to track this one down, I'll let you know if I do.

[cboulay]

Try this

install_name_tool -change "libpsmoveapi.3.dylib" "@rpath/libpsmoveapi.3.1.0.dylib" libpsmoveapi_tracker.dylib
install_name_tool -id "@rpath/libpsmoveapi_tracker.3.1.0.dylib" libpsmoveapi_tracker.dylib
install_name_tool -id "@rpath/libpsmoveapi.3.1.0.dylib" libpsmoveapi.dylib

I haven't tried in a while but I couldn't figure out how to get cmake to do this correctly. I know it's possible, but I just don't know the steps.

[DoctorZurk]

I succesfully compiled PSMOVE API with OpenCV and motion tracking enabled on OSX. Examples are working, controller pairing and PS eye are working. I tried to compile with the "XCode" option to generate the XCode project. It went fine (a part the missing RPATH complain).

Compiling the XCode project returns me this error on this line in file camera_control.c: cc->capture = cvCaptureFromCAM(cc->cameraID); No member named cameraID in struct_CameraControl I tried to fix that by putting cc->cameraRuntimeID instead, and it gave a new error: #include "malloc.h" File not found Changed into #include <malloc/malloc.h> Now it compiles, but I have this link errors:

 "_camera_control_set_exposure", referenced from:
      _psmove_tracker_set_exposure in libpsmoveapi_tracker_static.a(psmove_tracker.o)
      _psmove_tracker_adapt_to_light in libpsmoveapi_tracker_static.a(psmove_tracker.o)
  "_strncpy_s", referenced from:
      _camera_control_get_identifier in libpsmoveapi_tracker_static.a(camera_control.o)
ld: symbol(s) not found for architecture x86_64

I tried to force the X86 architecture, beause I'm aware of the 32bit pseye driver problem, but I got still the same link error.

What I'm trying to do is a OSC server (coded in unity for a non-ui executable), so I don't need the oculus / OVR "bindings". I would just need to send motion tracker and motion controller sensors data out through OSC. It would be great to have the Unity plugin not linked with OVR anymore, but with the new internal VR plugins, if really needed. I say this because I want to use the motion tracker but NOT for tracking directly in a unity game.

[HipsterSloth]

@DoctorZurk The psmove-unity5 plugin currently needs the camera tracker pose to put the psmove in the space of the HMD tracking camera. The only way to get the tracking camera pose presently is through the OVR plugin. The new VR functions don't appear to expose the tracking camera pose, only the HMD pose. Though please let me know if I missed something (I only briefly glanced at the UnityVR docs). There is a "VRDevice.GetNativePtr" method, but I have no idea what that returns on OS X. That said, if you can sit tight for a few weeks I'm working on something with cboulay that will replace the need psmove-unity5 that will allow you to use the psmove controller through SteamVR.

[thp]

What's the status of this? Looks like we have multiple issues put into one -- maybe it's possible that we split this up into separate issues and close this one? One issue per problem/bug makes it easier to track progress.

[DoctorZurk]

@HipsterSloth thank you for your time, I want to use the PS MOVE controller as a position tracker, not for using with PC Oculus. I just need position tracking and orientation tracking, and send those data (with TUIO probably) to the network. The cameras will be not attached to a rift pc but to a position tracking dedicated server. I wanted to do this with a unity app, I will probably make a console app, lighter, better. IMO also the PSMOVEAPI unity plug-in is now focused on the rift, which is ok, but in my opinion now, with new Unity version, should just focus on PSEYE/CAMERA tracking of the controllers in the space, abstracting by the headmount headtracking oculus or vive provide.

[thp]

I'm going to close this issue here, as the original issue seems to have been at least partially fixed. Please feel free to continue the discussion by creating a new issue and linking to this one for historic context.

[HipsterSloth]

@DoctorZurk I spun off a new issue in psmove-unity5 regarding decoupling the rift from psmove-unity5.