how to load the models in ionic 3.

[rajkishoreandia]

Fetch API cannot load file:///android_asset/models/face_recognition_model-weights_manifest.json. URL scheme "file" is not supported.

[justadudewhohacks]

Basically answered this here #73:

Hmm, that's odd. Usually browsers don't allow you to access files from your filesystem for security reasons, that's why the fetch request in model.load is failing in chrome (not sure why this works for you in firefox).

If you look at my examples in this repo, you can see that I state, which folders can be publicly accessed (in server.js). This way you can load your model from an url e.g. net.load('models/my-model'). You can do the same for your model.

With filepicker I mean, you can use an input element of type 'file', which is another way to browse and load files from your browser. There should be plenty of tutorials out there, explaining how to implement a file upload.

Another way is to monkey patch readFile as described here: #153 and then use net.loadFromDisk. I am not familar with Ionic so I don't know what kind of filesystem methods you have available to implement the patch.

[LeonardoDB]

I have the same problems importing the modules, I am using Ionic 3 and I am trying to use monkeyPatch.

faceapi.env.monkeyPatch({ readFile: filePath => Promise.all([ await faceapi.nets.tinyFaceDetector.loadFromUri('./models'), await faceapi.nets.faceLandmark68Net.loadFromUri('./models'), await faceapi.nets.faceRecognitionNet.loadFromUri('./models'), await faceapi.nets.faceExpressionNet.loadFromUri('./models') ]).then() });

[justadudewhohacks]

@LeonardoDB not sure why you are using loadFromUri inside your patch. You are supposed to monkey patch readFile using ionics specific file system API.

[LeonardoDB]

Thanks @justadudewhohacks, your help was helpful in troubleshooting

[yagancadorin]

@rajkishoreandia @justadudewhohacks

I am integrating face-api.js with an ionic3 project. With this, I was able to load the models for recognition.

1. Install cordova-plugin-file
2. Move all models to /src/assets/models
3. In your page.ts, declare:

let filePathRoot = this.file.applicationDirectory + 'www/assets/models/';

4. Create monkeyPatch

faceapi.env.monkeyPatch({
       readFile: filePath =>
         new Promise(resolve => {
           this.file.resolveLocalFilesystemUrl(filePath).then(res => {
             console.log(filePath);
             if (res.isFile){
               let fileExtension = filePath.split("?")[0].split(".").pop();
               let fileName = filePath.split("?")[0].split("/").pop();
                 if (fileExtension === "json") {
                   this.file.readAsText(filePathRoot, fileName).then((text) => {
                     resolve(text);
                   });
                 } else {
                   this.file.readAsArrayBuffer(filePathRoot, fileName).then((arrayBuffer) => {
                     resolve(new Uint8Array(arrayBuffer));
                   });
                 }
             }
         });
       }),
      Canvas: HTMLCanvasElement,
      Image: HTMLImageElement,
      ImageData: ImageData,
      Video: HTMLVideoElement,
      createCanvasElement: () => document.createElement("canvas"),
      createImageElement: () => document.createElement("img")
     });

5. Load models from Disk

await faceapi.nets.tinyFaceDetector.loadFromDisk(filePathRoot);
await faceapi.nets.faceRecognitionNet.loadFromDisk(filePathRoot);

6. Enjoy!

let resultsRef = await faceapi.detectAllFaces(image, new faceapi.TinyFaceDetectorOptions());
alert(JSON.stringify(resultsRef));

Hope this helps. Any adjustments or improvements, please pass on to the community!

[CodingApp]

@yagancadorin Thank you for sharing this, i have a query in this that do you have an example reference to this code ? If not please explain this if you can 'Move all models to /src/assets/models'.

Thank you.

[AnkitS221]

Please can anyone explain how to integrate FaceApi.js library in an ionic 3 projecct and how to use it? Sharing some code would definitely help

[yagancadorin]

@yagancadorin Thank you for sharing this, i have a query in this that do you have an example reference to this code ? If not please explain this if you can 'Move all models to /src/assets/models'.

Thank you.

I will try to make it easier. After installing the plugin, follow the steps:

1. Download JS file, move to /src/assets/js/ and include into your /src/index.html

[...]
<script src="cordova.js"></script>
<script src="assets/js/face-api.min.js"></script>
[...]

2. Download models and move all files to your /src/assets/models/

3. Declare faceapi in your /src/pages/home/home.ts

[...]

declare var faceapi : any;

@Component({
  selector: 'page-home',
  templateUrl: 'home.html'
})

export class HomePage { }

[...]

4. Create and call async loadmodel function

[...]

  async loadModels(){
    // set path to load models
    let filePathRoot = this.file.applicationDirectory + 'www/assets/models/';

    // faceapi settings
    faceapi.env.monkeyPatch({
      readFile: filePath =>
        new Promise(resolve => {
          let fileExtension = filePath.split("?")[0].split(".").pop();
          let fileName = filePath.split("?")[0].split("/").pop();

          this.file.resolveLocalFilesystemUrl(filePathRoot + fileName).then(res => {
            if (res.isFile){
                if (fileExtension === "json") {
                  this.file.readAsText(filePathRoot, fileName).then((text) => {
                    resolve(text);
                  });
                } else {
                  this.file.readAsArrayBuffer(filePathRoot, fileName).then((arrayBuffer) => {
                    resolve(new Uint8Array(arrayBuffer));
                  });
                }
            }
        });
      }),
     Canvas: HTMLCanvasElement,
     Image: HTMLImageElement,
     ImageData: ImageData,
     Video: HTMLVideoElement,
     createCanvasElement: () => document.createElement("canvas"),
     createImageElement: () => document.createElement("img")
    });

    // load models for recognition
    await faceapi.nets.tinyFaceDetector.loadFromDisk(filePathRoot);
    await faceapi.nets.faceRecognitionNet.loadFromDisk(filePathRoot);
    await faceapi.nets.faceLandmark68Net.loadFromDisk(filePathRoot);
    await faceapi.nets.faceExpressionNet.loadFromDisk(filePathRoot);
  }

[..]

5. Use your global faceapi variable normaly in code to create faceMatcher and detect/recognize faces.

[...]

let detections = await faceapi.detectAllFaces(shotCam, new faceapi.TinyFaceDetectorOptions()).withFaceLandmarks().withFaceDescriptors().withFaceExpressions();

[...]

[rajkishoreandia]

@justadudewhohacks @yagancadorin thanks for the response.I am now implementing the same in ionic 3 .but my models are not getting loaded from file path.i am getting following error. ERROR Error: Uncaught (in promise): [object Object]. FileError--->code: 1message: "NOT_FOUND_ERR"proto: Object Thanks in advance!

[rajkishoreandia]

Face detection is not happening @justadudewhohacks @yagancadorin After loading the models if i pass the image into face Api // load models for recognition await faceapi.nets.tinyFaceDetector.loadFromDisk(filePathRoot); await faceapi.nets.faceRecognitionNet.loadFromDisk(filePathRoot); await faceapi.nets.faceLandmark68Net.loadFromDisk(filePathRoot); await faceapi.nets.faceExpressionNet.loadFromDisk(filePathRoot), await faceapi.nets.ssdMobilenetv1.loadFromDisk(filePathRoot) console.log('after loading detection models') // displaying the fetched image content const myImg = document.getElementById('myImg') const detections = await faceapi.detectAllFaces(myImg).withFaceLandmarks().withFaceDescriptors() console.log('detections',detections) prints nothing in console

[lvallejo85]

Este error me da luego de compilar para ionic 6 ios