hitsthings / ptamjs

WIP!! Doesn't work yet
MIT License
1 stars 2 forks source link

Finishing this project #1

Open ThorstenBux opened 3 years ago

ThorstenBux commented 3 years ago

Hi @hitsthings, how are you? I just read your latest blog entry and am amazed at how many parallels we have roughly looking at our professional work life. But also family life. But I leave that for another day.

I'm a passionate WebAR developer and Co-founder of the WebARKit organization on GitHub (non-profit currently). We aim to be the go-to place for OpenSource Augmented Reality on the Web.

Looking for a starting point to implement surface tracking, I came across PTAM and with that to PTAMJS. I made it run and cleaned up this 8-year-old JavaScript code and feel a bit dusty right now ;). There is still a lot to do to bring it up to modern JavaScript. However, in an interest to move fast I thought I'd reach out to you and ask for help implementing the missing bits for a PoC of PTAMJS. Humbly speaking I consider myself a good (Web-) Developer and WebAR expert, however, when it comes to Computer Vision I still have a lot to learn. If you would have time to push forward on this project or to guide me along the way I would appreciate it very much. In fact, I already appreciate that you took the time to read my long message, thank you.

Kind regards and best wishes to NSW (does your new home still count as Sydney?) I lived in Sydney at some point in 2008. Anyway, all the best from Christchurch, NZ.

Thorsten

hitsthings commented 3 years ago

Hey mate! Thanks a ton for messaging me!

I really, really wanted to get this through when I started, but there were a lot of hurdles and other priorities got in the way. With three kids and a fledgling business, I'd be lying if I said things had changed, but I would be more than happy to put in a couple hours a week to help out if you want to run with it. To be clear, I'm also not a Computer Vision expert! I was just trying to translate the white paper into code to the best of my ability. Initially I wanted to use it with a quadcopter back in the day when Nodecopters were all the rage to make an autonomous one.

The first thing I tried was to take the C PTAM library itself and just add Node bindings to it. However I had a lot of problems getting it to compile on my computer. At the time I was using a Windows machine, and the library is built for Linux, so there were a lot of missing dependencies. I'm not sure what your goal is, but in the days of Windows Subsystem for Linux, the C library might be a better bet?

This repo was my second attempt, using the white paper to try to rebuild the library in JS. I think the main thing that stopped me was just running out of time, but there were also performance concerns and general difficulty for me in understanding some of the math (quaternions and matrices still make my head spin). Some of them might be solved by the new-fangled SharedArrayBuffer which will drastically reduce IO since the mapping and tracking threads can pass data around without so much overhead.

Let me know if you have any questions I could help with! Keen to hear what you had in mind for updating this! Like I said, I'd love to help, but also won't be able to put major hours into it. If there's a small piece you could use help with, I'm happy to take it on.

Cheers, Adam

ThorstenBux commented 3 years ago

Hi Adam,

thank you so much for your reply.

In one sentence: I want to get surface detection/tracking working in a web browser.

I looked into C PTAM and am considering to transpile it using Emscripten. Looking at it from a very high view, it has a lot of dependencies which are rather old so I assume it will give me a very hard time to transpile it.

I'd prefer to have a pure JS/TypeScript approach to get it to work.

However, looking at your project I lack the handle as to where to actually start. I could get the debug script and the test script working and I can see that it produces feature points. I'm assuming next would be to create a map from those points, but the map script states todo: https://github.com/hitsthings/ptamjs/blob/master/lib/map.js#L11

Also, this one here: https://github.com/hitsthings/ptamjs/blob/master/lib/tracker.js#L127 doesn't return a pose estimate.

These are basically the two places where I'm stuck as of now.

Eventually, I'd like to remove the Node.js dependency to be able to run it solely inside the browser. But that is a future goal. First is, make it work ;)

So, yes whatever assistance/time you can find to support me is highly appreciated.

Cheers Thorsten