search

mlcommons / mobile_models

MLPerf™ Mobile models
https://mlcommons.org/en/groups/inference-mobile/
Apache License 2.0
24 stars 10 forks source link

Add super resolution models and dataset #51

Closed anhappdev closed 1 year ago

anhappdev commented 1 year ago

The model is pl_f32b5 and the images are in raw rgb8 format. Since GitHub has a size limit of 100 MB, I used only 20/25 images for the lite dataset, which is used for the performance test. In case someone needs it, here is the full dataset with 25 images: snusr_lr_full.zip snusr_hr_full.zip

The result is posted here: https://github.com/mlcommons/mobile_app_open/pull/613#issue-1454410457

github-actions[bot] commented 1 year ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

anhappdev commented 1 year ago

LGTM. It looks models are not updated though. Latest models are supposed to have depth-to-space at the end (swap clamping (min+ max, or min+relu in TFLite) and depth-to-space)

Thanks for the catch. I updated the models.

fatihcakirs commented 1 year ago

The model is pl_f32b5 and the images are in raw rgb8 format. Since GitHub has a size limit of 100 MB, I used only 20/25 images for the lite dataset, which is used for the performance test. In case someone needs it, here is the full dataset with 25 images: snusr_lr_full.zip snusr_hr_full.zip

The result is posted here: mlcommons/mobile_app_open#613 (comment)

Hi @anhappdev , are these in landspace model. I also tried visualizing them (via matlab):

row=540; col=960; fileID = fopen('03a53ed6ab408b9f.rgb8','r'); A1 = fread(fileID, [col row], 'uint8=>uint8'); A2 = fread(fileID, [col row], 'uint8=>uint8'); A3 = fread(fileID, [col row], 'uint8=>uint8'); A1 = A1'; A2 = A2'; A3=A3'; Af = cat(3, A3, A2, A1) imshow(Af)

which kinda gave me a tree in landscape mode but quite right (shown below). Can you provide a way to visualize these rgb8 files.

image

anhappdev commented 1 year ago

@fatihcakirs All the images are rotated to landscape if it was in portrait. You can use ImageMagick to convert them from RGB back to JPEG: convert -size 960x540 -depth 8 rgb:04113e7d2f21171e.rgb8 04113e7d2f21171e.jpeg

freedomtan commented 1 year ago

The model is pl_f32b5 and the images are in raw rgb8 format. Since GitHub has a size limit of 100 MB, I used only 20/25 images for the lite dataset, which is used for the performance test. In case someone needs it, here is the full dataset with 25 images: snusr_lr_full.zip snusr_hr_full.zip The result is posted here: mlcommons/mobile_app_open#613 (comment)

Hi @anhappdev , are these in landspace model. I also tried visualizing them (via matlab):

row=540; col=960; fileID = fopen('03a53ed6ab408b9f.rgb8','r'); A1 = fread(fileID, [col row], 'uint8=>uint8'); A2 = fread(fileID, [col row], 'uint8=>uint8'); A3 = fread(fileID, [col row], 'uint8=>uint8'); A1 = A1'; A2 = A2'; A3=A3'; Af = cat(3, A3, A2, A1) imshow(Af)

which kinda gave me a tree in landscape mode but quite right (shown below). Can you provide a way to visualize these rgb8 files.

image

@fatihcakirs it seems you are dealing with 3 planes. Nope, it's rgbrgbrgb....

in Jupyter notebook, with Pillow, 

from PIL import Image

with open("/tmp/sr/dataset/HR_raw/03a53ed6ab408b9f.rgb8", mode="rb") as f:
    contents = f.read()
Image.frombytes('RGB', (1920, 1080), contents, 'raw')

Then you can see the image.

fatihcakirs commented 1 year ago

Thanks @freedomtan

Hi @anhappdev, RE: 100 MB issue, will it help to convert the images to .png format which does lossless compression? This should bring the total size below 100 MB in my experience.

anhappdev commented 1 year ago

Hi @anhappdev, RE: 100 MB issue, will it help to convert the images to .png format which does lossless compression? This should bring the total size below 100 MB in my experience.

The original images are in .png but the app currently cannot process .png so we decided to convert it to raw RGB since .jpeg has some accuracy issue. Please see the discussion here for more detail https://github.com/mlcommons/mobile_app_open/issues/612