[Documentation] Image struct is not correct in Livebook Examples

[andersbjorkland]

Running the code as detailed in the examples-livebook, aka:

image = Kino.Input.read(image_input)

# Build a tensor from the raw pixel data
image =
  image.data
  |> Nx.from_binary(:u8)
  |> Nx.reshape({image.height, image.width, 3})

Nx.Serving.run(serving, image)

This will produce the following error: ** (KeyError) key :data not found in: %{format: :rgb, width: 224, height: 128, file_ref: {:file, "..."}}

Solution Extract the image_data explicitly and pass it to Nx.from_binary:

%{file_ref: file_ref, format: :rgb, height: height, width: width} = Kino.Input.read(image_input)

# Load binary (rgb) data
image_data = 
  file_ref 
  |> Kino.Input.file_path() 
  |> File.read!()

# Build a tensor from the raw pixel data
image =
  image_data
  |> Nx.from_binary(:u8)
  |> Nx.reshape({height, width, 3})

Nx.Serving.run(serving, image)

As this produces the same expected tensor, it will also work in the Manual inference section:

inputs = Bumblebee.apply_featurizer(featurizer, image)
outputs = Axon.predict(resnet.model, resnet.params, inputs)

id = outputs.logits |> Nx.argmax() |> Nx.to_number()
resnet.spec.id_to_label[id]

[jonatanklosko]

This is a recent Livebook/Kino change, the example is already adjusted on main (ref), just not released yet :)

[andersbjorkland]

Ah, great. I just reviewed the issues and didn't see that. Good job on keeping ontop of it!