search

tensorflow / tfjs

A WebGL accelerated JavaScript library for training and deploying ML models.
https://js.tensorflow.org
Apache License 2.0
18.41k stars 1.92k forks source link

Mobilenet predictions are different on different devices #3966

Closed juliiami closed 1 year ago

juliiami commented 4 years ago

System information

  1. Windows 7, GPU: image Browser version: Google Chrome Version 84.0.4147.135 (Official Build) (64-bit)

Current behavior

Model predictions are different on 2 devices: 1. image

2. image

Checked also with tf.ENV.set("WEBGL_PACK", false), it doesn't influence on the predictions.

lina128 commented 3 years ago

@annxingyuan Do you know why inference result can be different on different device/os versions?

annxingyuan commented 3 years ago

Hi @juliiami - would you mind visiting https://js.tensorflow.org/debug/ on both platforms and pasting your results in this ticket? Thanks!

juliiami commented 3 years ago

Hi @juliiami - would you mind visiting https://js.tensorflow.org/debug/ on both platforms and pasting your results in this ticket? Thanks!

Hi @annxingyuan. The results look the same On the 1st device: image On the 2nd: image

sunbinyuan commented 3 years ago

I am having the same issue also where I observe different results across different devices with the same OS and browser for a completely different model. I am also able to replicate what @juliiami mentioned with his mobile net:

System 1 with i7-4720HQ and GTX965M on Chrome 89.0.4389.82: image

System 2 with i7-9750H and RTX2070 MaxQ on Chrome 89.0.4389.82: Untitled

I have already read online that it might be due to how the browser handles the image processing but that could be wrong. Also on my model, the results are inconsistent enough that the top prediction is different for the two systems. Any pointers would be appreciated.

murphycj commented 3 years ago

I also observed the same issue after visiting https://juliiami.github.io/tfjs_example/index.html. The results shown here https://js.tensorflow.org/debug/ are the same on all browsers though.

Browser Device OS Probability of Egyptian cat
Chrome - Version 89.0.4389.90 (Official Build) (x86_64) Macbook Pro Mac OS 11.2.2 0.589
Safari - Version 14.0.3 (16610.4.3.1.4) Macbook Pro Mac OS 11.2.2 0.687
Safari iPhone X iOS 14.4.2 0.680
Chrome - v90.0.4430.216 iPhone X iOS 14.4.2 0.680
Chrome - v81.0.4044.138 Galaxy s9+ Android 10; SM-G965U Build/QP1A. 190711.020 0.65

I actually came to this thread because I am observing a similar issue for an audio classification model I run in the browser with the latest version of tfjs. I am observing significantly different predictions for audio in certain browsers/devices. I think (but am still investigating) that the issue is due to tfjs because I saved the data right before it went into tfjs on those devices the model is performing poorly, and I then ran it through my python version of the tensorflow model and it performs as expected.

shmishra99 commented 1 year ago

Hi @juliiami, Apology for the late response. May I know if your issue has been resolved or still you are looking for the solution ? Thank You!

google-ml-butler[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.

google-ml-butler[bot] commented 1 year ago

Closing as stale. Please @mention us if this needs more attention.

google-ml-butler[bot] commented 1 year ago

Are you satisfied with the resolution of your issue? Yes No