hotg-ai / rune

Rune provides containers to encapsulate and deploy edgeML pipelines and applications
Apache License 2.0
133 stars 15 forks source link

Need simple procbloc that converts u8s to f32 #220

Open Ge-te opened 3 years ago

Ge-te commented 3 years ago

tf.js models are mostly unquantised, but RGB input is, to make runevm.js generic for images, we need a simple procbloc like this: PROC_BLOCK<U8[3, 224, 224], F32[3, 224, 224]> un-quantise hotg-ai/rune#proc_blocks/un-quantise

just converting all u8 ints into f32 floats like f32[0.0-1.0] = u8[0-255]/255

Michael-F-Bryan commented 3 years ago

This ties into something @Mi1ind ran into a while ago. Basically, we'll spend a bunch of time "casting" between tensors of different types where the tensor's contents don't really change value (e.g. 10_u32 goes to 10.0). At the moment we're handling this in a fairly ad-hoc fashion (make a new proc block for every cast), but in the long run we should come up with a better/more general/builtin way of doing it.

CC: @kthakore

Mi1ind commented 3 years ago

@Ge-te tested this out - We can use the existing image-normalization proc_block. Setting mean = 0.0, standard deviation = 255.0 would normalize u8 to f32 ([0,1]).

Runefile.yml for the proc_block would look like this:

normalize:
    proc-block: "hotg-ai/rune#proc_blocks/image-normalization"
    inputs:
      - image
    outputs:
      - type: F32
        dimensions:
          - 3
          - 224
          - 224
    args:
      rgb:
        - 0.0
        - 255.0