How to load typescript type definitions for this?

[TheInvoker]

Issue Description

It is saying node is not a property of faceapi.tf. And dispose is not a property of faceapi.tf.

Steps to Reproduce

import faceapi from '@vladmandic/face-api';

export default class FACE_DETECTION {

    static async GetFaceBox(buffer:Buffer) {

        await faceapi.nets.ssdMobilenetv1.loadFromDisk('model'); // load models from a specific patch
        //await faceapi.nets.faceLandmark68Net.loadFromDisk('model');
        //await faceapi.nets.ageGenderNet.loadFromDisk('model');
        //await faceapi.nets.faceRecognitionNet.loadFromDisk('model');
        //await faceapi.nets.faceExpressionNet.loadFromDisk('model');
        const options = new faceapi.SsdMobilenetv1Options({ minConfidence: 0.1, maxResults: 10 }); // set model options
        //const buffer = fs.readFileSync('man.jpg'); // load jpg image as binary
        const decodeT = faceapi.tf.node.decodeImage(buffer, 3); // decode binary buffer to rgb tensor
        const expandT = faceapi.tf.expandDims(decodeT, 0); // add batch dimension to tensor
        const result = await faceapi.detectAllFaces(expandT, options); // run detection
        //.withFaceLandmarks()
        //.withFaceExpressions()
        //.withFaceDescriptors()
        //.withAgeAndGender();
        faceapi.tf.dispose([decodeT, expandT]); // dispose tensors to avoid memory leaks
        // eslint-disable-next-line no-console
        if (result.length > 0) {
            var box = result[0].box;
            var data = { x:box.x, y:box.y, width:box.width, height:box.height };
            return data;
        }

        throw new Error("No face detected");
    }
}

Expected Behavior No typescript errors

**Environment

• Module version? 1.7.2
• Built-in demo or custom code? demo
• Type of module used (e.g. js, esm, esm-nobundle)? js
• Browser or NodeJS and version (e.g. NodeJS 14.15 or Chrome 89)? 16.13.2
• OS and Hardware platform (e.g. Windows 10, Ubuntu Linux on x64, Android 10)? Windows 11
• Packager (if any) (e.g, webpack, rollup, parcel, esbuild, etc.)? none

[vladmandic]

I'll take a look mid next week, currently traveling.

[vladmandic]

just catching up on items - to clarify, you're NOT using a bundler, this is pure TypeScript that you're transpiling using tsc? in that case, you need to import * as tf from '@tensorflow/tfjs-node'; before face-api - this is documented as tfjs-node cannot be pre-bundled.

and just to avoid any possibility of tsc making a mess of platform detection, import specific version of face-api import * as faceapi from@vladmandic/face-api/dist/face-api.node.js';`

and if tsc doesn't automatically recognize typedefs, do faceapi.tf = tf after the imports (it should not be needed in 99% of cases, but i don't know your environment);

[TheInvoker]

I am not using any bundler since I'm just dealing with server side code. It is pure typescript compiling only.

Also I am getting an error here

Could not find a declaration file for module '@vladmandic/face-api/dist/face-api.node.js'. 'C:/Users/me/Desktop/main/node_modules/@vladmandic/face-api/dist/face-api.node.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/vladmandic__face-api` if it exists or add a new declaration (.d.ts) file containing `declare module '@vladmandic/face-api/dist/face-api.node.js';`ts(7016)

[TheInvoker]

As well with this line

const result = await faceapi.detectAllFaces(expandT, options);

It shows an red underline on expandT saying

const expandT: tf.Tensor<tf.Rank>
Argument of type 'Tensor<Rank>' is not assignable to parameter of type 'TNetInput'.
  Type 'Tensor<Rank>' is not assignable to type 'Tensor4D'.
    Types of property 'shape' are incompatible.
      Type 'number[] | [number, number] | [number, number, number, number] | [number, number, number] | [number] | [number, number, number, number, number] | [number, number, number, number, number, number]' is not assignable to type '[number, number, number, number]'.
        Type 'number[]' is not assignable to type '[number, number, number, number]'.
          Target requires 4 element(s) but source may have fewer.

[vladmandic]

most of the errors come due to fact that tfjs is still compiled using tsc 3.x which is pretty much obsolete - so a lot of its typedefs are broken. but here's a full workaround that doesn't produce any errors:

import * as fs from 'fs';
import * as tf from '@tensorflow/tfjs-node';
import * as faceapi from '@vladmandic/face-api';

const modelUrl = 'node_modules/@vladmandic/face-api/model';

async function main() {
  await faceapi.nets.ssdMobilenetv1.loadFromDisk(modelUrl);
  await faceapi.nets.faceLandmark68Net.loadFromDisk(modelUrl);
  await faceapi.nets.ageGenderNet.loadFromDisk(modelUrl);
  await faceapi.nets.faceRecognitionNet.loadFromDisk(modelUrl);
  await faceapi.nets.faceExpressionNet.loadFromDisk(modelUrl);
  const options = new faceapi.SsdMobilenetv1Options({ minConfidence: 0.1, maxResults: 10 });
  const buffer = fs.readFileSync('face.jpg');
  // faceapi.tf.node only exists in nodejs environments and not in browsers, so its not present in typedefs
  // you can use string index instead of property to get tsc to ignore it
  const decodeT = faceapi.tf['node'].decodeImage(buffer, 3);
  const expandT = faceapi.tf.expandDims(decodeT, 0);
  const result = await faceapi
    // tfjs is compiled using old typescript v3
    // so when using new tsc such as 4.7 or 4.8 it cannot correctly infer types when there is type inheritance involved
    // e.g, it even complains for difference between tf.Tensor4D and tf.Tensor<Rank.R4> which are exactly the same
    // best to just tell faceapi this is valid input and it will take care of types as needed
    .detectAllFaces(expandT as faceapi.TNetInput, options)
    .withFaceLandmarks()
    .withFaceExpressions()
    .withFaceDescriptors()
    .withAgeAndGender();
  // anywhere there is faceapi.tf, you can use tf directly when in nodejs environment
  // so it bypasses face-api generated typedefs and uses native tfjs typedefs instead
  tf.dispose([decodeT, expandT])
  console.log(result);
}

main();

[TheInvoker]

I get an red underline under faceapi.tf['node'] saying:

Element implicitly has an 'any' type because expression of type '"node"' can't be used to index type 'typeof tf'.
  Property 'node' does not exist on type 'typeof tf'.ts(7053)

[vladmandic]

that is not an error in default tsc configuration - if your tsconfig.json settings are very aggressive, there will be many more. and like i said, you can use tf.node instead of faceapi.tf.node - but then you'll have to deal with tfjs v3 typedefs which can be even more messy (again, depends on your tsconfig settings)

all-in-all, i provided notes in the code above, its really up to you how you want to use it - there is not much more i can do

[TheInvoker]

I updated it to this to fix that error

{
    "compilerOptions": {
        "outDir": "dist/",
        "rootDir": "./src",
        "sourceMap": false,
        "noImplicitAny": false,
        "noImplicitThis":true,
        "noImplicitReturns": true,
        "moduleResolution": "Node",
        "esModuleInterop":true,
        "resolveJsonModule": true,
        "module": "ESNext",
        "target": "ESNext",
        "lib": [
            "ESNext"
        ],
        "allowJs": false,
        "alwaysStrict": true,
        "strict": true,
        "typeRoots": [
            "node_modules/@types",
            "node_modules/@vladmandic/types"
        ]
    },
    "include": [
        "src/**/*.mts"
    ],
    "exclude": [

    ]
}

But when compiling I get these errors now. I think it has something to do with the typeroots

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/engine/training.d.ts:144:22 - error TS2420: Class 'LayersModel' incorrectly implements interface 'InferenceModel'.
  Types of property 'inputs' are incompatible.
    Type 'SymbolicTensor[]' is not assignable to type 'ModelTensorInfo[]'.
      Type 'SymbolicTensor' is not assignable to type 'ModelTensorInfo'.
        Types of property 'shape' are incompatible.
          Type 'Shape' is not assignable to type 'number[]'.
            Type 'number | null' is not assignable to type 'number'.
              Type 'null' is not assignable to type 'number'.

144 export declare class LayersModel extends Container implements tfc.InferenceModel {
                         ~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:17:5 - error TS2411: Property 'input_shape' of type 'Shape | undefined' is not assignable to 'string' index type 'PyJsonValue'.

17     input_shape?: Shape;
       ~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:18:5 - error TS2411: Property 'batch_input_shape' of type 'Shape | undefined' is not assignable to 'string' index type 'PyJsonValue'.

18     batch_input_shape?: Shape;
       ~~~~~~~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:19:5 - error TS2411: Property 'batch_size' of type 'number | undefined' is not assignable to 'string' index type 'PyJsonValue'.

19     batch_size?: number;
       ~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:20:5 - error TS2411: Property 'dtype' of type 'keyof DataTypeMap | undefined' is not assignable to 'string' index type 'PyJsonValue'.

20     dtype?: DataType;
       ~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:21:5 - error TS2411: Property 'name' of type 'string | undefined' is not assignable to 'string' index type 'PyJsonValue'.

21     name?: string;
       ~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:22:5 - error TS2411: Property 'trainable' of type 'boolean | undefined' is not assignable to 'string' index type 'PyJsonValue'.

22     trainable?: boolean;
       ~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:23:5 - error TS2411: Property 'input_dtype' of type 'keyof DataTypeMap | undefined' is not assignable to 'string' index type 'PyJsonValue'.

23     input_dtype?: DataType;
       ~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:46:5 - error TS2411: Property 'inbound_nodes' of type 'NodeConfig[] | undefined' is not assignable to 'string' index type 'PyJsonValue'.

46     inbound_nodes?: NodeConfig[];
       ~~~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/training_config.d.ts:31:5 - error TS2411: Property 'metrics' of type 'string[] | { [key: string]: string; } | undefined' is not assignable to 'string' index type 'PyJsonValue'.

31     metrics?: MetricsIdentifier[] | {
       ~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/training_config.d.ts:34:5 - error TS2411: Property 'weighted_metrics' of type 'string[] | undefined' is not assignable to 'string' index type 'PyJsonValue'.

34     weighted_metrics?: MetricsIdentifier[];
       ~~~~~~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/training_config.d.ts:35:5 - error TS2411: Property 'sample_weight_mode' of type '"temporal" | undefined' is not assignable to 'string' index type 'PyJsonValue'.

35     sample_weight_mode?: SampleWeightMode;
       ~~~~~~~~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/training_config.d.ts:36:5 - error TS2411: Property 'loss_weights' of type 'LossWeights | undefined' is not assignable to 'string' index type 'PyJsonValue'.

36     loss_weights?: LossWeights;
       ~~~~~~~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/types.d.ts:90:5 - error TS2411: Property 'config' of type 'T' is not assignable to 'string' index type 'PyJsonValue'.

90     config: T;
       ~~~~~~

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/layers/core.d.ts:99:5 - error TS2411: Property 'seed' of type 'number | undefined' is not assignable to 'string' index type 'PyJsonValue'.

99     seed?: number;
       ~~~~

Found 15 errors in 5 files.

Errors  Files
     1  node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/engine/training.d.ts:144
     8  node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/topology_config.d.ts:17
     4  node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/training_config.d.ts:31
     1  node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/keras_format/types.d.ts:90
     1  node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-layers/dist/layers/core.d.ts:99

Do you know how to adjust the tsconig.json to fix this?

[vladmandic]

typescript changed how it handles string enums a lot since v3, so those are exactly one kind of typedef incompatibilities of tfjs itself that i was mentioning.

easiest is probably to set skipLibCheck so 3rd party libraries such as @tensorflow/tfjs-node are not checked for internal validity.

or disable noImplictAny so tf['node'] does not report error.

[TheInvoker]

Thanks, the skipLibCheck worked. Are there any plans to upgrade to typescript 4 compilation later?

[vladmandic]

Yes, work is in progress.

[vladmandic]

closing the issue as resolved, but feel free to post on this thread with any further questions.