Using zxing-js in a React Native project

[DSMikeNW]

Describe the bug I have been trying to use zxing-js in a react native project, digging thru the Issue reports I found this two: https://github.com/zxing-js/library/issues/148 https://github.com/zxing-js/library/issues/53

The first one relates to its use on Node, this is the way I have currently implemented zxing-js in my project, however while I have reached the point where the code runs without errors, I see the same problem that issue was meant to solve. That is each time I give the reader a valid image code it throws the:

NotFoundException: No MultiFormat Readers were able to detect the code.

The second one was someone who had the same question as I currently have but did not follow thru and the issue was dropped unsolved.

To Reproduce This is the code as it currently is in my project:

import { NativeModules } from 'react-native';
import  ImagePicker  from 'react-native-image-picker';

import { Buffer } from 'buffer';
const jpeg = require('jpeg-js');
const base64js = require('base64-js');
const { MultiFormatReader, BarcodeFormat, DecodeHintType, RGBLuminanceSource, BinaryBitmap, HybridBinarizer } = require('@zxing/library/esm5');

const options = {
  title: 'Select Avatar',
  customButtons: [{ name: 'fb', title: 'Choose Photo from Facebook' }],
  storageOptions: {
    skipBackup: true,
    path: 'images',
  },
};

let readQRFromImageFileAndLink = function () {

    ImagePicker.launchImageLibrary(options, (imageInfo) => {

      if (imageInfo.didCancel) {
        console.log('User cancelled image picker');
      } else if (imageInfo.error) {
        console.log('ImagePicker Error: ', imageInfo.error);
      } else if (imageInfo.customButton) {
        console.log('User tapped custom button: ', imageInfo.customButton);
      } else {
        decodeFile(imageInfo);

      }
    });
}

let decodeFile = async (image) => {
    global.Buffer = Buffer; // very important
    let buffer = Buffer.from(image.data, 'base64');

    let width  = image.width;
    let height = image.height;
    let source = { uri: 'data:image/jpeg;base64,' + image.data };

    let rawImageData = jpeg.decode(buffer);

    let hints = new Map();
    const formats = [
        BarcodeFormat.QR_CODE, 
        BarcodeFormat.DATA_MATRIX,
        BarcodeFormat.PDF_417,
        BarcodeFormat.AZTEC
    ];
    hints.set(DecodeHintType.POSSIBLE_FORMATS, formats);
    hints.set(DecodeHintType.TRY_HARDER, true);

    let reader = new MultiFormatReader();
    reader.setHints(hints);

    let luminancesUint8Array = new Uint8ClampedArray(rawImageData.data.length);
    for (let i = 0; i < rawImageData.data.length; i++) {
        // tslint:disable-next-line:no-bitwise
        luminancesUint8Array[i] = ((rawImageData.data[i * 4] + rawImageData.data[i * 4 + 1] * 2 + rawImageData.data[i * 4 + 2]) / 4) & 0xFF;
        }

    let luminanceSource = new RGBLuminanceSource(luminancesUint8Array, width, height);

    console.log(luminanceSource);

    let binaryBitmap = new BinaryBitmap(new HybridBinarizer(luminanceSource));

    let qrCode = reader.decode(binaryBitmap);

    console.log(qrCode);

    if (qrCode) {
        return JSON.parse(qrCode.getText());
    } else {
        console.error("failed to decode qr code.");
    }//*/

  };

module.exports = {
    readQRFromImageFileAndLink
}

Expected behavior I expected to at least be able to detect and read the currently supported codes.

Smartphone (please complete the following information): I'm currently using the default configuration recommended for react native on its website and an emulator with this settings:

• Device: Pixel 2 API 28
• OS: Android 9.0 x86
• 1080x 1920 xxhdpi
• Graphics: Automatic
  • Boot Option: Quick Boot
  • Multi-Core CPU : 2
  • RAM: 1536 MB
  • VM heap: 256 MB
  • Internal Storage: 6144 MB
  • SD card: Studio-managed 512MB

[odahcam]

First of all: I edited your issue so the code is now properly highlighted.

Second: looks like you have two issues, so let's take them one-by-one.

NotFoundException: No MultiFormat Readers were able to detect the code.

This is an expected behavior even if it's called exception. It was meant to tell users that no code was found, but was implemented in a terrible way. So, it represents no troubles at all, it's just the way the library is telling you it did not find anything in the given image.

I expected to at least be able to detect and read the currently supported codes.

Here I believe the problem is no matter what you give to the reader, no code is found, right? If so, you gotta make sure you create the BinaryBitmap right. Now I'm not familiar with the technique you're using, but I did help a user that was using the same jpeg-js library you're using here and we could successfully get decoded results. That means we do have a tested way to create BinaryBitmaps using this library, so if you're not, I will ask you to take a look in here: https://github.com/zxing-js/library/issues/289#issuecomment-612105311

That said I'll wait for follow up to continue, thanks for well documented issue.

[DSMikeNW]

I reviewed the thread and it gave me the idea of using each of the readers instead of the MultiFormatReader hopping to get a more detailed error instead of the NotFoundException: No MultiFormat Readers were able to detect the code.

I got the next error:

 WARN  Possible Unhandled Promise Rejection (id: 1):
FormatException
CustomError@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:193345:32
Exception@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:193290:30
FormatException@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:195642:45
getProvisionalVersionForDimension@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:204531:44
processFinderPatternInfo@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:205639:83
decode@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:203941:83
decodeFile$
tryCatch@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25623:23
invoke@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25798:32
tryCatch@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25623:23
invoke@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25699:30
http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25729:19
tryCallTwo@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:26987:9
doResolve@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:27151:25
Promise@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:27010:14
callInvokeWithMethodAndArg@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25728:29
enqueue@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25733:157
async@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:25749:69

__invokeCallback@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:2692:23
http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:2387:34
__guard@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:2604:15
invokeCallbackAndReturnFlushedQueue@http://10.0.2.2:8081/index.bundle?platform=android&dev=true&minify=false:2386:21
invokeCallbackAndReturnFlushedQueue@[native code]

I'm guessing either the ImagePicker is not giving me the base64 in the correct format or the Buffer is somehow messing with it thus the FormatException

[odahcam]

In that case we should reverse engineer those data to see what we have. Try to get those values and try working on them in order to make them a visible image again and you will be able to visually see if the values being passed are right.

[DSMikeNW]

Okay, I double checked the base64, the data in the Buffer and the luminanceSource everything up to that point seems to be working as expected and I was able to reverse engineer the data back to the image I'm using as an example.

My only guess is that there is something wrong with the BinaryBitmap, but I have no idea how to check if it's being created correctly.

BinaryBitmap:

{"binarizer": {"buckets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "luminances": [0], "matrix": null, "source": {"dataHeight": 119, "dataWidth": 119, "height": 119, "left": 0, "luminances": [Uint8ClampedArray], "top": 0, "width": 119}}}

I checked the Uint8ClampedArray and it seems to have the correct data from the luminanceSource.

[odahcam]

If the luminance source is ok there's almost no possibility BinaryBitmap is wrong. Do you got the error with any code you try to scan?

[DSMikeNW]

Okay, so I went thru all the image files we are using as test cases and found that around 10% of the files are actually read correctly, 60% are showing the error Error: SOI not found and 30% are showing the FormatException.

I'm going to dig thru the ones that are working correctly and see what's the difference to the others.

[odahcam]

Alright, I'll wait for updates. For the 60% failing, try enabling some custom encoder or decoder, you can find info on how to do that on our issues.

[DSMikeNW]

Okay, so after a good amount of research, checking the file formats and contents. It seems that the tool used to generate the codes was generating invalid jpg files, after trying out a different one I got a 100% success rate. I'm sorry for bothering you with this issue.

One last question, is there a package you recommend for png files that you would recommend to pair with zxing?

[odahcam]

No problem! For other formats I do use Sharp, you can find examples in the tests code. Although I don't know if it works in React Native.

[DSMikeNW]

Thank you, I will check it out.

[odahcam]

You're welcome. Please feel free to open new issues or share your discoveries.

[Andriiklymiuk]

@odahcam what tool did you used for rn then?

[odahcam]

I do not remember, it was a long time ago. Did you try @lovell/sharp?

[Andriiklymiuk]

Not yet, but thanks @odahcam