integrate various dlib-based addons to use a common library wrapper.

[roymacdonald]

Hey chris, hope you are doing well. I also have an addon for dlib. mine is ofxDLib and your´s ofxDlib. I think that we could integrate both and have just one addon for dlib. let me check the differences and I'll come back with an idea of an approach for such. what do you think?

best

[bakercp]

Hey @roymacdonald, I'm definitely up for it combining forces. We're using mine in a course I'm teaching and I just made it public to make it easier for my students to use. It's not ready for mass consumption -- I'm currently working on setting up type_traits to seamlessly use ofPixel types without converters in dlib.

In terms of related efforts, there is also ofxLearn @genekogan (general wrapping of ML algorithms) and ofxFaceTracker2 @HalfdanJ (general wrapping of FaceTracking/pose, replacement for ofxFaceTracker), and yours (non-dnn training and tracking generally) and mine, which is building out the new DNN capabilities. I've also focused on making it run with various hardware / library configurations on Linux / OSX (see scripts and addons_config.mk) and setting up data / model downloading (see also scripts).

[roymacdonald]

good to know were on the same boat! :) To be honest I think that your addon is a lot better than mine, as yours has all this config scripts. I'm aware of those other addons. My initial intention was to have a kind of wrapper like ofxCv where it mostly transforms data back and forth. I see that there is similar stuff on yours. I think that providing an implementation for moving data around would prove to be useful for the other addons as well as all the configuration stuff. Although I think that it is a good idea to keep this addons appart otherwise it would become a huge thing to have all merged into one. As I said, I'll come back with an idea for how to merge both. best!

[HalfdanJ]

Im also totally into removing dlib from ofxFaceTracker2, and use this instead. In ofxFaceTracker2 i got apothecary scripts for Android.

[roymacdonald]

@HalfdanJ great! good to know that you have those android scripts to! :) best

[genekogan]

i'd be interested also in participating in this. ofxLearn is wrapping dlib but it's an old version, and by now it makes sense to have all dlib stuff in one house. i've not looked carefully at the new deep neural net functionality, but if it is CUDA-based, could be really useful to work it all into one environment. in ml4a, we are using a combination of multiple libraries -- ofxLearn, ofxDarknet, ofxFaceTracker2, etc.

[bakercp]

Cool. @genekogan yes, dlib has a really nice template-based DNN API now and it's all CUDA accelerated if you compile w/ cuda -- you can even train with multiple graphics cards. That's why I built the scripts / setup the addon_config.mk to allow the users to be able use it more easily across platforms / hardware. In our class setting, there are 0 students with cuda compatible laptops, but we've installed GTX 1070/80 in a bunch of our lab machines running Ubuntu.

Anyway, I've been using this (along with some of your ml4a resources) in my course this semester, mostly because I felt given the experience level of my students staying within openFrameworks would be a more efficient classroom move, since its a hybrid computer vision ML course. Anyway, dlib's new DNN stuff is quite fast and well documented and there's a very active community happening around it.

I'd propose not to put all of the functionality into one place, but simply have ofxDlib be the shared library. Then ofxLearn, ofxFaceTracker2, ofx... (some other stuff I'm writing that hasn't been named) will all just include an ofxDlib as dependency.

In my opinion the ofxDlib would just have a bunch of examples that mirror the original dlib example + show how it integrates with native openFrameworks types (ofPixels, etc). That way people can more easily jump back and forth between oF / dlib.

I've also got ci working and we can use that to build "distro" versions to prevent users from having to compile their own libs without committing binaries to the repository (this is how oF is now working on the master branch).

Anyway, my immediate plans for this are to finish and push the type-traits adapters for the ofPixels, ofBuffer types that should allow no-bridge interaction with dlib functions that take dlib::rgb_array, dlib::matrix etc. While it will likely help with speed to eliminate copies, it will also allow us to use dlib in its unwrapped form, which is really nice because dlib has some amazing, well-documented examples.

And, @HalfdanJ if you hadn't noticed, I used your apothecary scripts as a starting point for mine. Cheers!

[genekogan]

this sounds like a good strategy to me.

i'm curious how the video tracking would compare to the ContourTracker in openCV.

right now with the deep learning examples, i don't quite see anything that ofxDarknet doesn't already do very well. perhaps its more stable though and has less memory issues which can be tricky in darknet.

[bakercp]

@genekogan Since darknet is obviously really impressive -- I spent quite a while trying to get darknet into a reliable form for my students (and I got close using ofxDarknet) but I found the raw darknet lib to be finicky to configure, crash-prone on big data sets and I didn't want to spend a lot of time modifying it and gluing together all of the endless pull requests offering fixes that haven't been making it into the main repo.

Alternatively, I have found dlib to be very well written, documented and tested (and fast where it needs to be), so I'm def headed in that direction.

[roymacdonald]

Hey guys, great to hear that you're all interested. @bakercp strategy sounds good to me. Very much the same that I was thinking. I've updated my addon with a mix/mod of thing taken both form @bakercp 's addon and @HalfdanJ 's.

@bakercp how do you intend to implement the following

type-traits adapters for the ofPixels, ofBuffer types that should allow no-bridge interaction with dlib functions

My bag of C++ tricks falls short here! :$

I came across a very nasty problem while doing this update and came around with an ugly hack to fix it. Please guys take a look at it https://github.com/roymacdonald/ofxDLib/issues/8 I'd really appreciate some feedback.

Best.

[roymacdonald]

Hey, I fixed that nasty hack in dlib, made a PR to it and it is already merged. (The internet is awesome. Had a problem, I fixed it and it became available worldwide in less than an hour).
@bakercp I'm still intrigued about how you intend to make the adapters for ofPixels. Any clue I could get? :)

[roymacdonald]

Another thing, @kashimAstro is very interested on adding more dlib things into OF. He made a fork of my ofxDLib and added some more thing but he says he has more examples he could add. Here's out discussion https://github.com/roymacdonald/ofxDLib/issues/9

I would like to have everyone on the same track, as I think we are right now, so there is no unnecessarily spent energy by making the same thing twice. In this sense I think that is a good idea to get @kashimAstro on this wagon.

I'm really excited about all this collaboration and looking forwards for a great outcome.

All the best!

[genekogan]

@bakercp yeah darknet sometimes does not adapt well to a laptop, at least not with the big models. Lowering the batch size and subdivisions (sometimes to just 1) in the .cfg files helps a lot with getting it to load without running out of RAM. but yeah, it's not as easy.

this is great to hear about dlib's community. I haven't engaged with it for a few years, I didn't even realize there was much of one. I'm curious about what motivates them to do this stuff in C -- is the dlib community oriented towards low-level/hardware type applications? that seems the obvious reason to me. I'm also curious about what potential there is to run a CPU version on a Raspberry Pi.

[bakercp]

Hi all, see the seamless dlib / ofPixel example in the https://github.com/bakercp/ofxDlib/tree/master/example_image.

There are still some template cleanup things I want to do, but it's getting close and basically you can use most dlib functions now with raw ofPixels types. It's snazzy.

[bakercp]

I'm in the process now of converting the rest of my examples to using native ofPixels / etc.

At some point in the near future I'll also add support for ofImage, ofTexture, ofBaseHasPixels, ofBaseHasTexture but for now that's all easy enough to do manually if needed. Basically, I just really like ofPixels and ofTexture and I don't love ofImage because it's a little too magical sometimes 🦄

[kashimAstro]

Sorry @roymacdonald for delay,

I'm fell into a spiral of Easter celebrations.. keep me updated if you need helps of any kind.

good day to all

[roymacdonald]

@bakercp awesome!! I was trying to achieve the same but got caught while figuring out the dlib's generic_image and the pixel_traits. dlib is super templated so I was hopping from file to file trying to figure out how it works. Super elegant solution.

All the best!

[bakercp]

Hi all, lots of updates in the develop branch. See example_bridge_of_to_dlib and example_bridge_of_to_dlib_direct for examples of direct wrapping.

@HalfdanJ My branch of ofxFaceTracker2 uses ofxDlib library now.