javascriptdata / danfojs

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
https://danfo.jsdata.org/
MIT License
4.76k stars 209 forks source link

nodejs_kernel_backend.js #33

Closed burque505 closed 3 years ago

burque505 commented 3 years ago

Running Win10 Pro, node v14.13.0, danfojs-node@0.1.5

I think there are errors in this file (@tensorflow/tfjs-node/dist/nodejs_kernel_backend.js). I made some changes as shown below (as there is apparently no 'createTensor() function', I just changed those references to 'createOutputTensor', which was a blind guess), and stopped a few errors, but I'm not sure my changes are correct. I'm getting errors any time a 'slice' is required, e.g.:

C:\Users\me\Documents\Javascript\danfo>node dict.js
C:\Users\me\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:2567
            throw ex;
            ^

TypeError: Cannot read property 'slice' of undefined
    at new Tensor (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:1808:28)
    at Engine.makeTensorFromDataId (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:2817:17)
    at NodeJSKernelBackend.createOutputTensor (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\dist\nodejs_kernel_backend.js:139:28)
    at NodeJSKernelBackend.getInputTensorIds (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\dist\nodejs_kernel_backend.js:159:30)
    at NodeJSKernelBackend.executeSingleOutput (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\dist\nodejs_kernel_backend.js:192:73)
    at NodeJSKernelBackend.linspace (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\dist\nodejs_kernel_backend.js:1498:21)
    at C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:17877:69
    at C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:2705:55
    at C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:2551:22
    at Engine.scopedRun (C:\Users\HP6300\node_modules\@tensorflow\tfjs-node\node_modules\@tensorflow\tfjs-core\dist\tf-core.node.js:2561:23)

Here's 'dict.js' (code straight from 10 Minutes to Danfo-js which produced the errors) .

const dfd = require("danfojs-node")

dates = new dfd.date_range({ start: '2017-01-01', end: "2020-01-01", period: 4, freq: "Y" })

console.log(dates);

obj_data = {'A': dates,
            'B': ["bval1", "bval2", "bval3", "bval4"],
            'C': [10, 20, 30, 40],
            'D': [1.2, 3.45, 60.1, 45],
            'E': ["test", "train", "test", "train"]
            }

df = new dfd.DataFrame(obj_data)
df.print()

My edits to nodejs_kernel_backend.js:

    NodeJSKernelBackend.prototype.getInputTensorIds = function (tensors) {
        var ids = [];
        for (var i = 0; i < tensors.length; i++) {
            if (tensors[i] instanceof int64_tensors_1.Int64Scalar) {
                // Then `tensors[i]` is a Int64Scalar, which we currently represent
                // using an `Int32Array`.
                var value = tensors[i].valueArray;
                var id = this.createOutputTensor([], this.binding.TF_INT64, value); // var id = this.binding.createTensor([], this.binding.TF_INT64, value);
                ids.push(id);
            }
            else {
                var info = this.tensorMap.get(tensors[i].dataId);
                // TODO - what about ID in this case? Handle in write()??
                if (info.values != null) {
                    // Values were delayed to write into the TensorHandle. Do that before
                    // Op execution and clear stored values.
                    info.id =
                        this.createOutputTensor(info.shape, info.dtype, info.values); // this.binding.createTensor(info.shape, info.dtype, info.values);
                    info.values = null;
                }
                ids.push(info.id);
            }
        }
        return ids;

and

    NodeJSKernelBackend.prototype.executeEncodeImageOp = function (name, opAttrs, imageData, imageShape) {
        var inputTensorId = this.binding.createOutputTensor(imageShape, this.binding.TF_UINT8, imageData); //         var inputTensorId = this.binding.createTensor(imageShape, this.binding.TF_UINT8, imageData);
        var outputMetadata = this.binding.executeOp(name, opAttrs, [inputTensorId], 1);
        var outputTensorInfo = outputMetadata[0];
        // prevent the tensor data from being converted to a UTF8 string, since
        // the encoded data is not valid UTF8
        outputTensorInfo.dtype = this.binding.TF_UINT8;
        return this.createOutputTensor(outputTensorInfo);
    };

All help and comments greatly appreciated.

risenW commented 3 years ago

This could be caused by the node version try using node version 12.18.2.

Also, check this issue #20