Edge TPU module

[Rick-Jongbloed]

Hi,

just to let you know i;ve started to implement an EdgeTPU module for MLapi. Its running on a Raspberry Pi 4b and it works properly. I added a few configuration settings and the new module file. It's still in a alpha state as i would like to have it coexist with the current OpenCV implementation. If you would like me to cut the changes into small parts and send some pull requests, just let me know. My fork is at https://github.com/Rick-Jongbloed/mlapi

Log file:

DEBUG: Object Recognition requested DEBUG: get_file returned: ./images/f178ffe7-c457-4da3-9599-6cb36c75edc7.jpg DEBUG: Initializing MobileNet SSD v2 TFlife EdgeTPU model DEBUG: edgetpu_ssd_mobilenet_models:./models/mobilenet-ssd-edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite DEBUG: MobileNet SSD v2 EdgeTPU initialization (loading model from disk) took: 834.326 milliseconds DEBUG: MobileNet SSD v2 EdgeTPU detection took: 35.33070373535156 milliseconds INFO: object:person at [836, 137, 918, 405] has a acceptable confidence:0.953125 compared to min confidence of: 0.4, adding INFO: object:car at [303, 120, 611, 364] has a acceptable confidence:0.91015625 compared to min confidence of: 0.4, adding INFO: object:person at [227, 146, 351, 373] has a acceptable confidence:0.58203125 compared to min confidence of: 0.4, adding`

127.0.0.1 - - [03/Apr/2020 23:28:12] "POST /api/v1/detect/object HTTP/1.1" 200 - DEBUG: Object Recognition requested DEBUG: get_file returned: ./images/42b52941-8eae-4dbc-9f05-d5cf41a598e1.jpg DEBUG: MobileNet SSD v2 EdgeTPU detection took: 15.930500984191895 milliseconds INFO: object:person at [836, 137, 918, 405] has a acceptable confidence:0.953125 compared to min confidence of: 0.4, adding INFO: object:car at [303, 120, 611, 364] has a acceptable confidence:0.91015625 compared to min confidence of: 0.4, adding INFO: object:person at [227, 146, 351, 373] has a acceptable confidence:0.58203125 compared to min confidence of: 0.4, adding

[pliablepixels]

Very cool. Yes, PRs welcome. I have a google coral and it will be useful to me too.

[Rick-Jongbloed]

That's great :-). Do you already have a plan how to implement different modules? I think it would be best if a module could be a pluggable module/package which includes settings and an object detection class. It should be possible to use this package directly in zmes en mlapi. What do you think?

[Rick-Jongbloed]

Closed by error

[pliablepixels]

No, not really. I agree - I've long wanted to break up all the modules so that it is reusable by both mlapi and local installs. I'm generally not a good coder (not my regular profession), so I do stuff and then think "ah, I should have done it better" and by then it's too late. Feel free to modularize as fit!

[Rick-Jongbloed]

I understand, I'm not a coder by profession myself. The first thing I would like to do is to add the sections. This way the following structure can be setup:

[yolo]
enabled=true
config=./models/yolov3/yolov3.cfg
weights=./models/yolov3/yolov3.weights
labels=./models/yolov3/yolov3.labels

[yolo_tiny]
enabled=false
config=./models/tinyyolo/yolov3-tiny.cfg
weights=./models/tinyyolo/yolov3-tiny.weights
labels=./models/tinyyolo/yolov3-tiny.labels

[edgetpu_mobilenet_ssd_v1]
enabled=false
min_confidence=0.4
model=./models/mobilenet-ssd-edgetpu/ssd_mobilenet_v1_coco_quant_postprocess_edgetpu.tflite
labels=./models/mobilenet-ssd-edgetpu/coco_labels.txt

[edgetpu_mobilenet_ssd_v2]
enabled=false
min_confidence=0.4
model=./models/mobilenet-ssd-edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
labels=./models/mobilenet-ssd-edgetpu/coco_labels.txt

I also thought that it would be nice to expose every object dectection model to a different endpoint. Maybe we could define it here as well? (So the integration with zmes keeps working :-)) So:

[yolo]
enabled=true
config=./models/yolov3/yolov3.cfg
weights=./models/yolov3/yolov3.weights
labels=./models/yolov3/yolov3.labels
endpoint=/object    # so it's still compatible with zmes and other services

I think in a first modular version we should still just have all variables in one file to not make to too complication. We can split it up later. Also the mlapi.py file needs to support every module. However i would suggest that we move the model download logic to the specific module files.

What do you think about this idea?

Edit; I found some instructions on using plugin modules. I'll see if I can structure the current modules (opencv face detection, opencv object detection) into some kind of plugin. - https://packaging.python.org/guides/creating-and-discovering-plugins/

[pliablepixels]

I like the idea of keeping a single config file and if the models can have their own sections it doesn't break zmes too.

I did not understand endpoint=/object -> what does that mean?

Finally, if you find a good method of doing a plugin system, I am open to doing a breaking change if we agree its a good/flexible way for the future. My goal is to also implement a real time (per frame) analysis system in the future where we should be able to reuse our modularization.

[Rick-Jongbloed]

Regarding the endpoints I think I didn't understand it correctly. When looking at the code now i see there's one endpoint available (http://localhost:5000/api/v1/detect/object) and face detection is an option (http://localhost:5000/api/v1/detect/object?type=face).

If it would be possible to define a type for every model, in the config file (or use the section name...) You could instruct the API as below and the API would instruct the plugin to process the image. http://localhost:5000/api/v1/detect/object?type=face http://localhost:5000/api/v1/detect/object?type=yolo http://localhost:5000/api/v1/detect/object?type=yolo_tiny http://localhost:5000/api/v1/detect/object?type=edgetpu_mobilenet_ssd_v1 http://localhost:5000/api/v1/detect/object?type=edgetpu_mobilenet_ssd_v2 http://localhost:5000/api/v1/detect/object?type=edgetpu_face (or something)

With endpoint I meant the type of the object, however i think it would be best to use the name of the plugin there?

For the plugin framework I will do a bit of searching and will let you know. Lets first decide on the framework before any real change is made. This page looks like a good start: https://alysivji.github.io/simple-plugin-system.html

The realtime analysis system sounds good. Thats really what i'm missing at the moment, but implementing the detecton methods as plugins would be a nice start 👍

[pliablepixels]

It might be better to do this: http://server/api/v1/detect/<plugin>?<parameters>

Where <plugin> = face, object,gait,img2txt,... And <parameters> could be anything specific to that plugin including, say, model=yolo&type=tiny or model=ssd_net&whatever

Thoughts?

[Rick-Jongbloed]

Thats even better:-) it would also be nice to define default options for each endpoint, to ensure backwards compatibility?

[pliablepixels]

Sure. My thought is let’s do the right approach and get to backward compatibility later. I’m ok with the interface breaking as it’s still beta.

[Rick-Jongbloed]

Check. I'll try to make a poc, let's discuss further based on that code.

[Rick-Jongbloed]

I've been playing around with the instructions on https://www.guidodiepen.nl/2019/02/implementing-a-simple-plugin-framework-in-python/ and came up the following. This code automatically registers a class with the following name: plugins.<subfolder name>.<file name>,<class name>. As you can see below the classes register themselves and the init section of the script is loaded.

Looking for plugins under package plugins
    Found plugin class: plugins.identity.Identity_test
Initializing log
    Found plugin class: plugins.object_detection.yolo3.Object
    Found plugin class: plugins.face_recognition.face.cnn
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0

There's a global class definition where the main class is defined. I've changed the example a bit to match the current class definition:

class Plugin(object):
    """Base class that each plugin must inherit from. within this class
    you must define the methods that all of your plugins must implement
    """

    def __init__(self):
        self.description = 'UNKNOWN'

    # def perform_operation(self, argument):
    def detect(self, argument):
        """The method that we expect all plugins to implement. This is the
        method that our framework will call
        """
        raise NotImplementedError

Now I would like to discuss what methods there need to be in Plugin class. On a functional level, I'm thinking of:

• init: loads the model into memory, make sure there's a minimal load time when the first image is processed.
• parameters: defines the parameters bound to this plugin (currently in common_params.py file.
• models: defines the download method for the models
• detect: runs detection of an image.

is there anymore you can think of?

[pliablepixels]

From a method perspective:

• init : any init work
• detect: actual detection
• destroy: any cleanup
• describe: just a function that returns a text string about the plugin. Might be useful to show

Do we really need models/parameters? Should they not be implicitly invoked inside detect (or init)? To keep parameters simple, we can pass a params object to detect. Each plugin can mandate fields within that or not.

[Rick-Jongbloed]

Yes you are right, I don't think we need to expose the models and parameters. Is describe required as there's already a way to describe the plugin?

self.description = 'UNKNOWN'

I'll try to convert your current mlapi code into a plugin based part based on your method perspective.

[pliablepixels]

The describe() function is essentially a simple way for external programs to get access to self.description, especially if you instantiate a plugin but don't extend it. However, this is a very minor thing. We can keep extending the base later as needed (and what is logical). Yes, let's discuss on code. Note that when you extend, if there are similar functions in mlapi and ES, the ES always has the latest code (for example, I recently added a lot of error handling). So you may want to use the ES functions where applicable. (And hopefully, once we make these all common modules that can be installed with pip3, I won't have to keep patching both)

[pliablepixels]

@Rick-Jongbloed how are things going?

[Rick-Jongbloed]

Hi, I've been busy with other stuff due to the lockdown... I'll try to spend a weekend in may on this!

[pliablepixels]

closing this as EdgeTPU is done and moved to pyzm