ntedgi / node-efficientnet

tensorflowJS implementation of EfficientNet 🚀
https://www.npmjs.com/package/node-efficientnet
MIT License
257 stars 37 forks source link

replace `jimp` with `sharp`, support more image format, and better performance #35

Closed luoye-fe closed 3 years ago

luoye-fe commented 3 years ago

Sometimes need to analyze svg or webp file, but jimp not support.

luoye-fe commented 3 years ago

Maybe conflict with prev-pr, if so, i will fix them one by one.

ntedgi commented 3 years ago

Sometimes need to analyze svg or webp file, but jimp not support.

how many channels each pixel of this format as ? is it possible to represent them as simple RGBA ? or you need to make another convert

note : the first layer need at least 3 channels

also keep in mind the playground UI uploader doesn't support these types so we need to make changes in the angular app and test it

luoye-fe commented 3 years ago

At last, all image will be to raw, then toBuffer, and a buffer of length (width height channels), containing 8-bit RGB(A) pixel data

But it does need more test case, I'll add it later, ui I will also take a look

ntedgi commented 3 years ago

i ran npm run example in your branch and master it looks like there's is different prediction

master

[ { label: 'sports car, sport car', precision: 87.682247186149 }, { label: 'convertible', precision: 7.831910054231743 }, { label: 'car wheel', precision: 4.485842759619256 } ]

luoye-fe:sharp

[ { label: 'sports car, sport car', precision: 82.06160142748257 }, { label: 'car wheel', precision: 9.929226947224464 }, { label: 'convertible', precision: 8.009171625292971 } ]

can you please check again if the transformation to from jimp to sharp is accurate

also I added and example for all supported format lets test them

Screen Shot 2021-03-28 at 9 47 36
luoye-fe commented 3 years ago

ok, I will do more test

luoye-fe commented 3 years ago

basic image format test: image

more image format test: image

Except for the some heic picture, the test results of other picture formats are all good, This problem has bothered me for some days, Maybe you can help me see it together

test code https://github.com/luoye-fe/node-efficientnet/blob/sharp_test/test.ts

From some efficientnet article, preprocess input action is only center crop and resize to suitable resolution, maybe need other operations?

ntedgi commented 3 years ago

basic image format test: image

more image format test: image

Except for the some heic picture, the test results of other picture formats are all good, This problem has bothered me for some days, Maybe you can help me see it together

test code https://github.com/luoye-fe/node-efficientnet/blob/sharp_test/test.ts

From some efficientnet article, preprocess input action is only center crop and resize to suitable resolution, maybe need other operations?

the haic result look very strange , i took the operation procedure from this repo https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet

luoye-fe commented 3 years ago

merge upstream & reopen.