How does it work?

[mcsimenc]

Hi,

I am having trouble finding information about what IBEIS actually does. Is that information available anywhere? I have seen the color pictures and the extensive method list, looks very interesting, but nothing is said about what the modules are for.

Thanks, Matt

[Erotemic]

Good question, its not exactly clear. It was my first really big project, and it shows in the architecture, docs, and general layout. There are a lot of lessons learned in this repo, but I do believe that all the information to get it and its dependencies installed is in the super_setup.py file (perhaps much of it is outdated, but a clever developer could probably work with it).

IBEIS is software that I helped develop for animal identification during my PhD. It manages images and annotations on those image (which are usually around some sort of animal). It uses SIFT descriptor matching with KD-trees to find ranked lists of individuals and contains UIs to annotate them. It also contains work done by @bluemellophone, @hjweide, and @zmjjmz for object detection and other methods of animal identification.

It was first developed to be a standalone application, but a few years ago my advisor partnered with the the Wildme project and from then on this started to become more focused on image analysis and less focused on the user interaction needed to curate these computer vision algorithms. As such the Qt GUI in ibeis/gui began to become more and more of a developer debugging interface. This repo began to move towards a service based design where the Wildme/Wildbook software would send us a "query" image and we would run our ranking algorithms and return the top results.

I'm not planning on supporting this repo too much as code developed after 2016 unfortunately needed to be be moved to a private repo :(, however I can give you a quick rundown of the code structure:

Lets start with the top level sub-packages

• ibeis.control - contains the "ibs" controller and sqlite schema, as well as methods for interacting with the backend sqlite database.
• ibeis.algo - contains the "hotspotter" recognition algorithm and detection algorithm
• ibeis.dbio - handles importing other database formats into the internal SQL database as well as exporting subsets of the database
• ibeis.expt - code for managing multiple experiments with multiple configurations of the same algorithm (basically grid search for good hyperparams).
• ibeis.gui - code for the PyQt GUI
• ibeis.init - the first module imported, handles command line interfaces and initialization steps on the first run of the system. This works in conjunction with main_module.py
• ibeis.scripts - one-off scripts, usually experimental
• ibeis.other - do you like hacks? because this module has a lot of them. I'm not sure I can even begin to describe the horror that is this module. However, it is still tightly coupled with other parts of the system.
• ibeis.templates - helps with generation of Jupyter notebooks (at the time IPython notebooks) so the system could be demoed easilly
• ibeis.tests - a sparse suite of tests and test helpers
• ibeis.viz - matplotlib visualizations and interactions for the annotations, images, and algorithm results.
• ibeis.web - the code for running as a service for wildbook to talk to

As for the non-obvious top-level modules:

• _ibeis_object.py, annots.py, images.py - these are newer modules I made with the intention of making the API a bit simpler. They provide a vectorized and object oriented interface to data accessors.

• core_annots.py, core_images.py - these use the "depcache" (dependency cache), which is implemented in dtool. Each function here declares what it needs to compute a specific piece of information. They basically provide a declarative way to construct a lazily executable pipeline such that you can ask for a property (e.g. keypoints) of a "root" object (e.g.) an annotation, and it knows how to compute all the prereqs for the requested object (e.g. you need to extract and preprocess the chip of the annotation from its parent image before you compute its keypoints).

• new_annots.py - terribly named module. Has code for constructing data needed to run the SMK (selective match kernel) matching algorithm.

• annotmatch_funcs.py - Interface to SQL database dealing with pairwise information between annotations.

• params.py - CLI stuff

Hope this helps. Note that this was all from memory, so I may be off one a few things. Lots of the functions (but not all) do have docstrings / doctests though, so those are other places to look for help.

[bluemellophone]

Hello @mcsimenc!

We have been working behind the scenes (with the help of Microsoft) to update the IBEIS software and provide a consistent release schedule. This process includes 1) a new Docker image for easy deployment, 2) a better REST API with interfaces to review images, annotations, parts, and matches, and 3) better documentation and usage examples.

While this process is slow, we are making iterative progress towards a public release towards the end of the year. In the meantime, the software is being internally tested and used in our Wildbook platform for detection and identification on a wide range of species.

[VLucet]

@bluemellophone Just wondering if there are updates on that process. I am currently reviewing ML methods for CT data and I am curious about ibeis.

[bluemellophone]

@VLucet I recommend checking out Wild Me's WBIA project: https://docs.wildme.org/docs/developers/wbia/wbia_overview