opentrack integration

[sthalik]

Hey,

Are you interested in using opentrack/facetracknoir protocols, filters and trackers?

The code an easily be adapted (by me) to work without any kind of GUI, and with C API in order to get rid of name mangling issues.

[AndersMalmgren]

That sounds really awsome! How would it work? You run opentrack/facetracknoir in the background and the C API fetches the data for FreePIE?

Or would the API contain the entire software?

[sthalik]

No, either through DLL, itself without GUI (just be aware, parts of the code are GPL3), or a subprocess with shared memory mapping. The latter has an advantage of non-GPL3-ness.

API would contain the entire software without GUI, including, on a by-need basis, protocols and trackers.

[AndersMalmgren]

One other problem I see with first approach is config, you still need todo config from GUI, right? FreePIE has a mechanism for plugin settings, but its very basic.

We use GPL2 today because we have some libs under that license, I do not know if it wors with GPL3

[sthalik]

Not really. It's plain text using QSettings. You can use subprocess to get rid of GPL damage...

[AndersMalmgren]

Ok so the idea is to config through the Facetrack GUI, and then copy the settings to the FreePIE folder?

edit: Sorry, Im confusing Opentrack and Facetracknoir

[sthalik]

You're missing NPPriv_* stubs, for one. Also you link against a specific CRT version which is very meh.

Don't dllexport your own functions. Do NPPriv stubs. As for sig, just xor it with some crapola and you'll be fine. Anyhoo, good to stick a finger to the corporate man instead of cowering in fear of a lolsuit. Streisand effect and all that.

[sthalik]

Sorry meant to reply in the other issue.

You can get either the source, linked to in the other issue, or check the binary. There's a list of exported stuff. Don't ever link to specific CRT version, as stated, too.

---

As for UI, you can make your own configurator for select trackers. It could be simple and declarative, i.e. generate from a data structure describing data types and range.

[AndersMalmgren]

I think its best we think agile here, maybe you can do a POC and I can implement it in FreePIE edit: I'm talking about opentrack here :D

[sthalik]

It's C#, right?

Does it work with Mono without wine or Windows-only?

Can fork, do initial impl and send a pull request.

The blocker here is opentrack headless support, but it's rather trivial, as can do that without the main UI. The individual protocols and trackers don't use the main UI's symbols, the main UI uses them.

So the procedure:

• make libopentrack-headless.dll for access to trackers/protos by name, using 'extern "C"'
• integrate into freepie

I know C# pretty alright, even though prefer F#. If made some working class wrapping around yet-nonexistent libopentrack-headless.dll with P/Invoke glue and a clear api, will you be able to use it?

[AndersMalmgren]

The GUI wont work in Mono that i'm sure of, the console version might work but it has not been tested or reported to work. Also many of the plugins uses win32 API so those wont work.

Yes if you make it a C-lib I can invoke it from C#, you can also use C++ but you need to expose the api as C library

Another approach is that you expose the data using shared memory directly from the opentrack software, that way we can benefit from using your GUI etc

[sthalik]

What kind of UI do you use? Forms will work, only WPF won't.

I'm gonna use C++ with extern declarations. API won't expose internal details, just a bunch of functions with argument types like const char*, int, etc.

Exposing the whole UI doesn't imo make much sense, it won't execute in a batch manner like this. But you can use opentrack via freetrack protocol already. I don't like the latter idea, though.

Do you want to use filters? i.e. Accela mk3, as well as protocols, or just trackers? If just trackers, less work for me.

Finally, my idea is to make opentrack GUI use libopentrack.dll just the same, so that all consumers use it. Both FreePIE and opentrack itself on equal footing!

-sh

[AndersMalmgren]

Its WPF :P Also you might need to change libs in the lib folder to mono versions. We didnt use nuget because it did not work with VS2010 express.

Exposing trackers will be perfect!

A tip is to install a Virtual Machine with Window 7 / 8 and install VS2012 on that. Even if mono works its probably easier with native .NET 4.0

[sthalik]

Here's how it works as of last commit, as in:

https://github.com/opentrack/opentrack/blob/master/opentrack-api/opentrack.h

include <stdio.h>

include <time.h>

include "opentrack.h"

opentrack ctx = opentrack_make_ctx(argc, argv, NULL);

^ This thingie never returns a null pointer. The last argument is HWND cast to 'void*' of a window to parent with the headpose display, such as camera feed.

opentrack_tracker t = opentrack_make_tracker(ctx, "ht");

^ this can return a null pointer if name's not ok, or init fails for whatever reason.

opentrack_tracker_start(ctx, t);

^ need to call before receiving data

    if (opentrack_tracker_tick(t, headpose))

^ need to call damn thing frequently! Some trackers, such as face trackers, are stateful and as such, will 'lose track' a lot if they're not updated.

    {
        for (j = 0; j < 6; j++)
            printf("%f ", headpose[j]);
        printf("\n");
    }

}

^ update headpose display

opentrack_finalize_tracker(t);
opentrack_finalize_ctx(ctx);

^ free sysmem

Anything else you need or can you take if from here?

There's also tracker enumeration, check opentrack.h for details.

-sh

[AndersMalmgren]

Nice job, Do you have a binary i can test?

[sthalik]

Try this one:

http://ananke.laggy.pk/opentrack/opentrack-20131030.7z

Just keep in mind that debug info is in DWARF format, built using mingw-w64 from Linux.

[AndersMalmgren]

Thanks, Sorry I must admit I'm not that used to work in C / Cpp. Whats DWARF format? What calling convention should I use to invoke libopentrack-api.dll?

[sthalik]

The default calling convention, don't specify one. Probably cdecl, forgot which :P

DWARF format is debug info used instead of PDB. Only meaningful it you run into issues with the dll.

Of course give a ping if you run into issues, or get it working :)

Oh can you tell if you get video feed done via Control.Handle?

[AndersMalmgren]

Cool! Sadly cant try tonight :/ But first thing this weekend

I can try giving it a HWnd, but currently FreePIE does not support custom GUIs for plugins. It only support very basic settings to be set through a plugin settings mechanism we built. Can opentrack be operated completely dark?

[sthalik]

will add dark when home cheers good work ethic

[sthalik]

If you pass (void*)-1 as the parent HWND, it'll function with no frame visible. See latest revision.

[AndersMalmgren]

Sorry was super busy this weekend, didn't have time to code a single line :/ Is the null pointer HWND stuff built in the version found here (I havent beeen able to build the source with VS2012)

http://ananke.laggy.pk/opentrack/

[sthalik]

Yes, it's included. Add ((void*)-1) instead of nullptr, nullptr causes it to appear...

Get a8, it's even newer than that.

If you want ot build by yourself, I'm able to help.

[AndersMalmgren]

I must admit that I do not know how to define that in C#?

[sthalik]

cast -1L to intptr /phone

[sthalik]

I can provide you an API later, if can't do it, by writing myself PInvoke (fancy name for FFI) bindings. Always good to refresh the knowledge.

What is status? -sh

[AndersMalmgren]

I'm at work so cant try to invoke the dll now, but can try when I get home. It would be really good if you could create a wrapper for the dll in C# Mono, test it from a console app (Not FreePIE) when you have tested it and it works I can incorperate it in FreePIE, would save me alot of work since I'm not used to Opentrack.

I can do it, but I think it would save us alot of time if you could just create a small wrapper it could look like this

public class OpenTrackApi 
{
   public bool Init() 
   {
      //Calls opentrack_make_ctx, opentrack_make_tracker and opentrack_tracker_start
   }

   public HeadTrackingData Update() 
   {
      //Calls opentrack_tracker_tick
   }
}

[sthalik]

can add idisposable and dtor no biggie cheers

[AndersMalmgren]

Yes, adding a Dispose method or anything else to that class is perfectly fine :D But it's good if its public interface is as KIS as possible

[sthalik]

Sorry for delay. Real life happened. Lack of sleep, at that.

Already got VS Express 2013 installed in a VM, a new build using a working GNU toolchain, and so on.

I haven't forgotten or anything, it's just lack of time.

-sh

[AndersMalmgren]

oh, no worries, I know how hard it can be to find time for the hobby projects. My girlfriend does not understand how I can code all day at work and then want to code when I get home :D

[sthalik]

Dang, can't believe it's been almost 3 weeks already! :(

Can't promise to expect something soon, but will -try- to get some code running.

[AndersMalmgren]

No worries! I'm leaving for Thailand for a few weeks if you want the api tested in FreePIE please talk to my brother (CyberRascal on the mtbs forum) he can help you

[sthalik]

Any progress? If not, I can work on it some more at unspecified future time...

[sthalik]

Can't build the damn thing. Log follows:

5>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.Core\bin\Debug\FreePIE.Core.dll' could not be found 8>------ Build started: Project: FreePIE.Tests.Core.Plugins, Configuration: Debug Any CPU ------ 9>------ Build started: Project: FreePIE.Tests.Core, Configuration: Debug Any CPU ------ 8>C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(1635,5): warning MSB3270: There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "SlimDX, Version=4.0.11.43, Culture=neutral, PublicKeyToken=b1b0c32fd1ffe4f9, processorArchitecture=x86", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project. 8>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.GUI\bin\Debug\plugins\FreePIE.Core.Plugins.dll' could not be found 8>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.Tests.Test\bin\Debug\FreePIE.Tests.Test.dll' could not be found 9>C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(1635,5): warning MSB3245: Could not resolve this reference. Could not locate the assembly "Rhino.Mocks, Version=3.6.0.0, Culture=neutral, PublicKeyToken=0b3305902db7183f, processorArchitecture=MSIL". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors. 9>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.Core\bin\Debug\FreePIE.Core.dll' could not be found 9>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.GUI\bin\Debug\plugins\FreePIE.Core.Plugins.dll' could not be found 9>CSC : error CS0006: Metadata file 'C:\Users\Administrator\dev\FreePIE\FreePIE.Tests.Test\bin\Debug\FreePIE.Tests.Test.dll' could not be found 7> Restoring NuGet packages... 7> To prevent NuGet from downloading packages during build, open the Visual Studio Options dialog, click on the Package Manager node and uncheck 'Allow NuGet to download missing packages'. 7> All packages listed in packages.config are already installed. 7>C:\Users\Administrator\dev\FreePIE\FreePIE.GUI\Views\Script\ScriptEditorView.xaml(40,10): error MC3074: The tag 'CompletionPopupView' does not exist in XML namespace 'clr-namespace:FreePIE.GUI.CodeCompletion;assembly=FreePIE.GUI.CodeCompletion'. Line 40 Position 10. ========== Build: 1 succeeded, 8 failed, 0 up-to-date, 0 skipped ==========

[AndersMalmgren]

There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "SlimDX, Version=4.0.11.43, Culture=neutral, PublicKeyToken=b1b0c32fd1ffe4f9, processorArchitecture=x86", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.

This is strange. Have you changed any of the project settings?

[sthalik]

Didn't touch your solution's settings. Used visual studio 2013.

[AndersMalmgren]

I haven't gotten around to install vs13. It works for me in 12, will install 13 asap

[AndersMalmgren]

Builds for me in 2013 after a small change in one of the build steps

749cbc91c549fc2277888d6a7eeae5db8db012ae

[sthalik]

Confirm it builds, for now. Back to implementing FFI glue then.

[sthalik]

As C interface's more than enough on my part, won't participate in it further.

[AndersMalmgren]

I have too much on my table right now, I'm reopening this issue if someone want to have a go at it. An alternative is also to let opentrack read and write to FreePIE IO

https://github.com/AndersMalmgren/FreePIE/wiki/IO-Plugin

[sthalik]

Still here. At best can explain API specifics. Hacking C# ain't what it used to.

The opentrack-api code was removed, but it's a matter of reverting the removal.