Open touchhope opened 8 years ago
touchhope
commented
8 years ago OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat, file C:\builds\2_4_PackSlave-win32-vc12-shared\opencv\modules\core\src\matrix.cpp, line 323
touchhope
commented
8 years ago Regular update commit number d44e966 on 5 Aug 2015 is much more functional than commit number 65abaf6 on 8 Jul 2015 that I previously toughly solved on compiling errors under vs2015 community :
absolutely no compiling errors under vs2015 community
apparently very fast gesture recognition and 4 keys press detection during the 3 calibration steps
also udp channel messages sent and received between track_plus and win_cursor_plus processes
good base I think to try to understand what wrong in subsystem console instead of subsystem windows launch maybe (https://msdn.microsoft.com/en-us/library/f9t8842e.aspx)
I'm definitely not gifted enough on C++ coding language ! However I still try to change different parameters and still search on internet forums. I'm quite obstinated when I start to solve something !
JerryBocciJr
commented
8 years ago Keep up the good work! Hopefully, this summer I'll get enough slack that I'll be able to contribute to the codebase. C# is my current language of choice, but I cut my teeth on C/C++ years (decades?) ago, and now just need slack time to figure out what Lai was thinking when he abandoned ship - and possibly help fix/finish it.
touchhope
commented
8 years ago Hi Jerry !
Pleased to finally get in touch(plus ;-) ) with the famous Jerry Bocci Jr, also a kind of "touchhope" at https://www.kickstarter.com/projects/haptix/haptix-multitouch-reinvented/comments !
It was funny the other day when I've sent an email to australian backer Lachlan Pollock (friend of Maz), asking him to post you on Kickstarter a request of technical explanations on how you finally get it to work on your pc !
Indeed, as the frenchman that I am, I didn't understood on first read it was a joke ! In the minute I've sent another email to Lokki to say him I realize you were joking. He said you're probably a sarcastic man and of course you were ironic on that post !
Lai must be laughing at us seeing how stupid we are on not understanding his few commented code. Or maybe he is in a depressive phase so he is psychologically washed out after so big coding effort last year. Fortunately a band of stupid people can become intelligent if they communicate with each other. That's what we need : a good brain storming !
Lai is surely a brillant young engineer but, as he will grow old, he will be learning to better communicate with other people. No man is an island and what is more exciting in human life is surely what we can learn one from each other to go further together. You know like monkeys and in 2016 we are in the chinese year of monkey, aren't we ?
I invite Lai to reestablish dialog with us, people who still believe in your genious idea and project ! Come on Lai, thanks in advance man, you're the best :D ! You're mustn't be a dog and us neither, we're human beings !
I guess what is preventing win_cursor_plus process from being launched properly must be a silly thing such as "true" or "false", or "1" instead of "null" in some function.
As a csharp expert like Matthew H. Fogle, maybe you could explain us how a csharp exe is controlled ? Is there such thing as the manifest file for example ? (https://msdn.microsoft.com/en-us/library/1w45z383%28v=vs.110%29.aspx)
What about C:\touch_plus_source_code\track_plus_visual_studio\win_cursor_plus\ cs files, also in TUIO folder ?
Maybe some examples to examine at http://www.tuio.org/?software and http://blog.mosthege.net/2012/08/19/windows-8-touchinjection-with-c/ and http://touchinjector.codeplex.com/ where you suspect windows firewall to get implicated on port blocking (blue message from last commit in December) ?
To me there could also be a kind of debugging console mode that Lai has established to develop his code and which is preventing win_cursor_plus to get launched in windows mode : some lines somewhere to get commented or uncommented maybe ?
Just time to find !
I'm sure one day we'll all, even Lai, be renamed Tom, Jerry, ultimate mouse killers !
I'll be glad to read you also on http://gharbi.me/ractiv/ soon Jerry !
Keep being confident in finding a solution together !
Cheers,
Touchhope (or Christian or The Hopin'Frog or The Artist or Prince or the unbelivable ramblings spammer at http://gharbi.me/ractiv/ !)
touchhope
commented
8 years ago Hi all the seekers (and researchers) for functional solution for the touchplus device's software !
Well in the past few days, I didn't found much things :
If you don't want console to get shown, you just have to comment "// #define SHOW_CONSOLE" before compiling in file track_plus_core/daemon_plus/globals.h !
Then the 4 processes (daemon_plus.exe, menu_plus.exe, track_plus.exe and win_cursor_plus.exe) still get launched but without showing any console, like in the commit 950e348 on 6 Oct 2015 (with pretty circle jauge instead of 4 calibration steps with 3 gestures and 4 keys pressed), which indeed shows no console while running, but unfortunately with inert launched process "win_cursor_plus.exe", maybe not receiving any udp port message from track_plus.exe !
Well it would be cool to finally understand how and where in code udp messages are sent from track_plus process to win_cursor_plus process and its subprocess "MainWindow" as seen on taskmgr.
If it could help, for the specialists on csharp, here is the debugging errors tracking while testing a commit but I don't know which anymore, I've obtained the following errors from launched win_cursor_plus that just crashed :
"Unhandled exception: System.Windows.Markup.XamlParseException: The constructor invocation on the type 'win_cursor_plus.MainWindow' that matches the specified binding constraints threw an exception. ---> System.IO.DirectoryNotFoundException: Could not find a part of the path 'C: \ touch_plus_source_code \ build \ ipc'.
at System.IO . Error.WinIOError (Int32 errorCode, String maybeFullPath)
to System.IO.FileSystemEnumerableIterator1.CommonInit () to System.IO.FileSystemEnumerableIterator1..ctor (String path, String originalUserPath, String searchPattern, SearchOption SearchOption, SearchResultHandler`1 resultHandler, Boolean checkHost)
to System.IO.DirectoryInfo.InternalGetFiles (String searchPattern, SearchOption SearchOption)
to System.IO.DirectoryInfo.GetFiles (String searchPattern)
to win_cursor_plus.FileSystem.ListFilesInDirectory (String path) in C: \ touch_plus_source_code \ track_plus_visual_studio \ win_cursor_plus \ FileSystem.cs: line 34
win_cursor_plus.IPC.Update to () in C: \ touch_plus_source_code \ track_plus_visual_studio \ win_cursor_plus \ IPC.cs: line 69
to win_cursor_plus.IPC..ctor (String selfNameIn) in C: \ touch_plus_source_code \ track_plus_visual_studio \ win_cursor_plus \ IPC.cs: line 61
win_cursor_plus.MainWindow..ctor to () in C: \ touch_plus_source_code \ track_plus_visual_studio \ win_cursor_plus \ MainWindow.xaml.cs: line 123
--- End trace of inner exception stack ---
to System.Windows.Markup.WpfXamlLoader.Load (XamlReader XamlReader, IXamlObjectWriterFactory writerFactory, Boolean skipJournaledProperties, Object RootObject, XamlObjectWriterSettings settings, Uri baseURI)
to System.Windows.Markup.WpfXamlLoader.LoadBaml (XamlReader XamlReader, Boolean skipJournaledProperties, Object RootObject, XamlAccessLevel AccessLevel Uri baseURI)
to System.Windows.Markup.XamlReader.LoadBaml (Stream stream, ParserContext parserContext, Object parent, Boolean closeStream)
to System.Windows.Application.LoadBamlStreamWithSyncInfo (Stream stream, ParserContext pc)
to System.Windows.Application.LoadComponent (resourceLocator Uri, Boolean bSkipJournaledProperties)
to System.Windows.Application.DoStartup ()
to System.Windows.Application. <. ctor> b1_0 (Object unused)
to System.Windows.Threading.ExceptionWrapper.InternalRealCall (Delegate callback, Object args, Int32 numArgs)
to System.Windows.Threading.ExceptionWrapper.TryCatchWhen (Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
to System.Windows.Threading.DispatcherOperation.InvokeImpl ()
to System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext (Object state)
to System.Threading.ExecutionContext.RunInternal (ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
to System.Threading.ExecutionContext.Run (ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
to System.Threading.ExecutionContext.Run (ExecutionContext executionContext, ContextCallback callback, Object state)
to MS.Internal.CulturePreservingExecutionContext.Run (CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
to System.Windows.Threading.DispatcherOperation.Invoke ()
to System.Windows.Threading.Dispatcher.ProcessQueue ()
to System.Windows.Threading.Dispatcher.WndProcHook (IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean & handled)
to MS.Win32.HwndWrapper.WndProc (IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean & handled)
to MS.Win32.HwndSubclass.DispatcherCallbackOperation (Object o)
to System.Windows.Threading.ExceptionWrapper.InternalRealCall (Delegate callback, Object args, Int32 numArgs)
to System.Windows.Threading.ExceptionWrapper.TryCatchWhen (Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
to System.Windows.Threading.Dispatcher.LegacyInvokeImpl (DispatcherPriority priority, TimeSpan timeout, Delegate method, args Object, Int32 numArgs)
to MS.Win32.HwndSubclass.SubclassWndProc (IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
to MS.Win32.UnsafeNativeMethods.DispatchMessage (MSG & msg)
to System.Windows.Threading.Dispatcher.PushFrameImpl (DispatcherFrame frame)
to System.Windows.Threading.Dispatcher.PushFrame (DispatcherFrame frame)
to System.Windows.Application.RunDispatcher (Object ignored)
to System.Windows.Application.RunInternal (Window window)
to System.Windows.Application.Run (Window window)
win_cursor_plus.App.Main to () in C: \ touch_plus_source_code \ track_plus_visual_studio \ win_cursor_plus \ obj \ Release \ App.g.cs: Online 0"
touchhope
commented
8 years ago touchhope
commented
8 years ago Martin Kaltenbrunner have just answered my demand of help :
mkalten commented 12 hours ago • edited
"hi @touchhope, honestly I never heard of this device before ... looks interesting though. as far as I understand, the manufacturer left you with non-functional drivers and sdk? from what I saw quickly reviewing their code, they are using the CSHARP TUIO client in their software, which may be a hint that their driver actually sends touch data encoded as /tuio/2Dcur cursors.
you could try to use one of the available TUIO client demos for any platform or language and see if you can receive anything on their UDP port (which seems to be different from the standard TUIO UDP port 3333)
you could also try to update the TUIO CSHARP code, which would also be the right place for this discussion, since this here is the C++ version ... https://github.com/mkalten/TUIO11_NET
if you want to provide me with such a device, I can have a closer look of course
edit: it also seems that they modified/extended the TUIO protocol, which probably results in making it incompatible with standard TUIO software. what's exactly the function of the win_cursor_plus component?"
touchhope
commented
8 years ago touchhope
commented
8 years ago http://reactivision.sourceforge.net/
Absolutely interesting !
touchhope
commented
8 years ago A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction :
touchhope
commented
8 years ago Hi the touchhopers !
After testing a lot of commits, the most interesting to me is Regular update d679834 on 24 Jul 2015 https://github.com/Ractiv/touch_plus_source_code/commit/d6798349bd6b4d385c4bef170a068a914b99287f because it shows us how win_cursor_plus process is supposed to control TUIO cursor.
Indeed, when you compile this commit (a lot of dependencies are missing in its dependencies folder and must be completed with other dependencies comming from other later commits) you can get a medium window called "image_visualization_pointer_mapper" showing us the supposed behavior of TUIO cursor as seen on Alexandru Bauer's Youtube on v0.55b release without full available source code on github.
2 circles get superimposed as finger is coming to a centered small pointing area and distance to get in contact with virtual surface on keyboard is detected by pointer mapper system which is called from main.cpp in track_plus in this commit.
Unfortunately this commit isn't stable enough and you get crashing of deamon_plus and also track_plus process in few minutes.
But no TUIO cursor is launched through win_cursor_plus process and indeed nothing can be listened using https://github.com/gregharding/TUIOListener !
However I've set TUIO port to standard 3333 port instead of random TUIO port.
How could we send to TUIO cursor tracking data sent to window called "image_visualization_pointer_mapper" ?
How could it had worked on Alexandru Bauer pc as seen on Youtube : https://www.youtube.com/watch?v=ulFTmjiwmAA ?
To get a very stable commit without the tasklist command problem in software path, you can try the last commit before "processes.cpp stability fixes" commit 6b7e36e on 2 Nov 2015, "In progress: new algorithm for finger segmentation" https://github.com/Ractiv/touch_plus_source_code/commit/266d5e64dcdf8a0e19e44aa64059843017303f72.
The problem is that 4 points keyboard keys location calibration system seems to have been abandoned in the last commits (new circle initialization calibrating jauge) and we have to analyze how we can reconstruct logical pointer mapper system.
Indeed the notion of "palm point" seems to be important to inject tracking data to the win_cursor_plus process under csharp TUIO protocol.
So "Improved location of palm_point" on commit 2c0207a on 4 Nov 2015, post "processes.cpp stability fixes" commit must also be examined to try to understand how pointing circles get controled in "image_visualization_pointer_mapper" window https://github.com/Ractiv/touch_plus_source_code/commit/2c0207a2e0ddfe1e0ec9fd43dcca81f714f5b0fc.
At worst, we could restart from the beginning from Umar Nizamani work and programming the proposed mathematical solution "A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction" by Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu, and maybe mixing it with Matthew H. Fogle csharp proposal to get interaction the TUIO cursor !
So we need enraged newbies programmers to get it working with the minimal cursor function because it seems coding experts won't lose their precious time on that damn project !
What a pity that Lai Schwe isn't kind enough to share minimal information so that we must made a sort of retro-engineering to try to understand something making lot of tests !
Cheers !
Your desperate Touchhope still trying to find something interesting... until the hope will definitely died !
touchhope
commented
8 years ago
Hi everyone ! It's again Touchhope from http://gharbi.me/ractiv/ !
Well, with my slow old brain, I've only just understand that it was possible to get ancient states of touchplus source code from github, clicking on "307 commits", then on the update that you want and then on "Browse files" on the right, then on the green button 'Clone or download", for example at https://github.com/Ractiv/touch_plus_source_code/commits/master?page=9 from the very beginning.
After a lot of search on internet forums, to solve all compiling errors under vs2015 community from vs2013 code, (adding compiling options in properties under vs2015, adding several includes on top of a lot of the cpp files such as #include "stdafx.h", adding file in C:\External Storage\Dropbox\projects\touch_plus_source_code\track_plus_core\track_plus\imu.cpp, making all includes pointing to right absolute path dependencies, using Find And Replace Tool (https://findandreplace.codeplex.com/), I have zipped an ancient state of code from early 2015 July with all functionnal calibration steps and an udp channel opened for win_cursor_plus process which is launch in console mode unfortunately, in file win10vs2015touch_plus_source_code.zip, available on dropbox at https://www.dropbox.com/s/7oh7defgiwpe42z/win10vs2015touch_plus_source_code.zip?dl=0
To get it work on your win10, first install Touch+ Software.exe to get path C:\Program Files (x86)\Ractiv\Touch+ Software on your pc, then overwrite all its content with "build" folder content.
Don't forget to set your keyboard to English QWERTY instead of AZERTY to perform calibration steps ! Be patient while moving your fingers, and hold finger long time enough to let the software detect the key pressed on the keyboard !
Then you could make some code modifications under Visual Studio 2015 Community you'll find here : https://www.visualstudio.com/en-us/products/vs-2015-product-editions.aspx, loading track_plus.sln project file in C:\touch_plus_source_code\track_plus_visual_studio.
I hope it will help someone (Matthew H. Fogle maybe, or Umar Nizamani when he would be able to buy a new win10 pc, or Maz and his team on OpenCV in California, or someone else with better knowledge than me, a pure self-educated c++ dabbler and beginner) to produce some hybrid version under win10 and vs2015 (of old and last state of code from 1st of 2015 December).
Now the technical challenge is maybe to pick up some portions of code in the different states of code during 2015 year and to mix it adding it maybe to this basic version on win10 and vs2015, instead of vs2013 and win7 or win8 ?
I'd like to end with these words from backer Robert Sweetman one year ago : "As someone once said... No man is an island. Thus no great things come from someone who cannot communicate with their fellows as they remain forever doomed as legend in their own mind only. An inability to deliver on anything, promises, product etc. leads to further isolation. Seek first to understand before trying to be understood. Be of service to others ahead of expecting rewards for empty words. From the short interview text below you could drive a bus through the perceptual issues/potential roadblocks associated with this insular approach. I now formally abandon all hope of seeing anything from this venture. The smartest people in the world are usually sunk by their inability to communicate."
You're so right Robert ! Indeed what about Albert Einstein ? Was he able to communicate enough ? And Nicolas Tesla ? And Rudolf Diesel ? And Xue Lai (or Lai Schwe ?) And Darren Lim ? Surely, maybe too much complicated thoughts to get explain to other people !
Could someone tell me (Corey Manders maybe at IEEE or Dr. Andrea Tagliasacchi ?) why this win_cursor_plus process isn't launched properly to show its big green pointing circle ?
Indeed if you'll make a try under vs2015 with this old state of code from vs 2013, you'll need to first install "vs2015 nuget package manager", you'll find here : https://visualstudiogallery.msdn.microsoft.com/5d345edc-2e2d-4a9c-b73b-d53956dc458d and then, once installed in vs2015, going in the nuget manager in tools menu, click on browse thumb to search for missing "TCD.System.TouchInjection. 1.0.0" !
Cheers everyone and thanks in advance for any eventual help !
Touchhope
settings file loaded track_plus created message sent: menu_plus show stage menu_plus created Touch+ Camera:vid_1e4e pid_0107 Touch+ Camera found HD WebCam:vid_1bcf pid_2c17 camera opened = 1 settings file loaded message sent: menu_plus show notification Please wait:Initializing Touch+ Software lmmin start 17 -0.0625 102 27 => 8.8574512464 lmmin diag 17 -0.0625 102 27 => 38.937247358 o i lmpar prered ratio dirder delta pnorm fnorm p0 p1 p2 p3 0 0 7.5e-10 0.99 -136188 -0.99 3.894e+03 3.894e+03 3.257578520679308e+03 1350.89623 -9.37912809 -846073.533 1295.47822 0 1 9.8e-09 0.99 -1341.89 -0.99 3.894e+02 3.894e+02 3.234297096063896e+02 152.992915 -8.79677653 -83455.5088 156.868713 0 2 1.1e-07 0.99 -9.14679 -0.99 3.894e+01 3.910e+01 2.812345443854284e+01 31.9310619 -8.73832706 -6362.60825 41.7329552 0 3 0.75 0.47 -0.533259 0.023 3.894e+00 4.128e+00 9.895639494260942e+00 17.219583 -2.1818491 185.809731 27.2941152 0 4 16 0.07 1.20613 0.027 3.894e-01 3.942e-01 8.474394462928776e+00 16.9125343 -0.243465595 73.888238 26.9152316 1 0 3 0.071 0.999151 0.012 7.885e-01 8.084e-01 8.167136010881887e+00 16.798468 -0.453914014 69.1773574 26.8799825 2 0 1 0.085 1.03525 0.033 1.617e+00 1.621e+00 7.800304404582053e+00 16.8434865 -0.816411489 78.8717992 27.3255439 3 0 0.5 0.17 1.10073 0.072 3.242e+00 3.151e+00 7.021468914398151e+00 16.7437777 -1.47364135 93.821699 28.442294 4 0 0.25 0.41 1.14788 0.12 6.303e+00 5.898e+00 5.108576367012606e+00 15.8210873 -2.63903859 109.33949 30.7108406 5 0 0.0091 0.98 0.689894 -0.83 1.180e+01 1.183e+01 2.913732420989827e+00 12.6857568 -4.76178647 123.025422 35.5100193 6 0 0 0.94 0.992328 -0.94 1.180e+01 4.198e+00 7.431822867262537e-01 11.5896332 -3.89756675 119.762047 35.3019043 7 0 0 0.11 0.904534 -0.11 8.397e+00 2.837e+00 7.051821872474435e-01 10.8442948 -3.56552813 117.221326 36.8446364 8 0 0 0.011 0.999469 -0.011 5.674e+00 3.517e-01 7.011518348117394e-01 10.7594774 -3.5554181 116.899842 37.0814765 9 0 0 5e-06 1.04006 -5e-06 7.034e-01 1.130e-01 7.011500253813939e-01 10.7937891 -3.56917069 116.974485 37.0197881 10 0 0 1.2e-07 1.1806 -1.2e-07 2.260e-01 2.202e-02 7.011499742902622e-01 10.8013456 -3.57200733 116.984487 37.0083637 11 0 0 6.6e-09 1.28289 -6.6e-09 4.404e-02 7.173e-03 7.011499713402897e-01 10.8037166 -3.57290845 116.988223 37.0045488 12 0 0 5.4e-10 1.28086 -5.4e-10 1.435e-02 2.022e-03 7.011499710994058e-01 10.8043908 -3.57316295 116.98925 37.0034774 13 0 0 4.3e-11 1.30325 -4.3e-11 4.043e-03 5.704e-04 7.011499710799826e-01 10.8045809 -3.57323471 116.98954 37.0031751 14 0 0 3.7e-12 1.29113 -3.7e-12 1.141e-03 1.701e-04 7.011499710783136e-01 10.8046374 -3.57325608 116.989628 37.0030847 15 0 0 2.1e-13 1.68566 -2.1e-13 3.403e-04 3.441e-05 7.011499710781922e-01 10.8046494 -3.57326048 116.989643 37.0030669 16 0 0 1.5e-13 0.485264 -1.5e-13 6.881e-05 3.853e-05 7.011499710781660e-01 10.8046613 -3.57326521 116.989667 37.0030458 17 0 0 2.6e-15 2.47817 -2.6e-15 7.706e-05 9.574e-07 7.011499710781638e-01 10.8046618 -3.57326532 116.989666 37.0030456 18 0 0.00061 9.5e-15 1.00485 4.1e-15 1.915e-06 1.915e-06 7.011499710781605e-01 10.8046612 -3.57326509 116.989665 37.0030467 19 0 0 1.9e-15 -0.913541 -1.9e-15 3.830e-06 3.611e-06 7.011499710781610e-01 10.8046603 -3.57326467 116.989662 37.0030487 lmmin outcome (1) xnorm 73.6149 ftol 6.66134e-15 xtol 6.66134e-15 lmmin final 10.8046612 -3.57326509 116.989665 37.0030467 => 0.70114997108
a: 10.8047 b: -3.57327 c: 116.99 d: 37.003 pose estimator initialized message sent: win_cursor_plus open udp channel message_received start daemon_plus0 message_received get toggles daemon_plus1 message sent: menu_plus get toggles 1011005 message_received open udp channel 60445 track_plus0 udp port is 60445 message sent: menu_plus show window message sent: menu_plus show wiggle exposure is 10 message sent: menu_plus show point message sent: menu_plus show calibration message sent: menu_plus show calibration next step 0 complete message sent: menu_plus show calibration next step 1 complete message sent: menu_plus show calibration next step 2 complete message sent: menu_plus show calibration next step 3 complete message sent: menu_plus show stage calibration finished
message received open udp channel win_cursor_plus0 message sent: track_plus open udp channel 60445 bound to UDP port 60445