webpack fails on npm run start

[kmclaugh]

When following the instructions here

I get the following error when trying to run locally (npm run start).

const components = getBuildableComponents();
                   ^

TypeError: getBuildableComponents is not a function
    at Object.<anonymous> (/Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/webpack.config.js:7:20)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
    at Function.Module._load (internal/modules/cjs/loader.js:723:14)
    at Module.require (internal/modules/cjs/loader.js:848:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at WEBPACK_OPTIONS (/Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/node_modules/webpack-cli/bin/utils/convert-argv.js:114:13)
    at requireConfig (/Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/node_modules/webpack-cli/bin/utils/convert-argv.js:116:6)
    at /Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/node_modules/webpack-cli/bin/utils/convert-argv.js:123:17
    at Array.forEach (<anonymous>)
    at module.exports (/Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/node_modules/webpack-cli/bin/utils/convert-argv.js:121:15)
    at Object.<anonymous> (/Users/kevinmclaughlin/projects/ds-community-connector-tutorial2/barchart/node_modules/webpack-dev-server/bin/webpack-dev-server.js:84:40)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
Command failed: webpack-dev-server --open

[mattkizaric-leveragelab]

I had the same issue earlier this week. It seems like there was something wrong with the distribution package. I was able to fix this by cloning this library to a different location, going into the packages/dscc-scripts directory and running the build command. You can then point all of the scripts from the package.json dscc-gen made to the packages/dscc-scripts/index.js file. Bit of a hack, but good for the quick fix.

[kmclaugh]

Thanks for the heads up. I ended up doing what you described and then copying the new util.js file generated by your suggested clone over the node_modeules/@google/dscc-scripts/build/viz/util.js file generated by the installation in the tutorial.

Anyway, the fix seems to be to update the `viz/util.js' file with the below code.

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.getComponentIndex = exports.getBuildableComponents = exports.validateConfigFile = exports.validateManifestFile = exports.validateBuildValues = void 0;
/**
 * @license
 * Copyright 2018 Google LLC
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
const validate = require("@google/dscc-validation");
const fs_1 = require("fs");
const args_1 = require("../args");
const util_1 = require("../util");
exports.validateBuildValues = (args) => {
    const components = exports.getBuildableComponents();
    const devBucket = process.env.npm_package_dsccViz_gcsDevBucket;
    if (devBucket === undefined) {
        throw util_1.invalidVizConfig('gcsDevBucket');
    }
    const prodBucket = process.env.npm_package_dsccViz_gcsProdBucket;
    if (prodBucket === undefined) {
        throw util_1.invalidVizConfig('gcsProdBucket');
    }
    const devMode = args.deployment === args_1.DeploymentChoices.PROD ? false : true;
    const gcsBucket = devMode ? devBucket : prodBucket;
    const manifestFile = 'manifest.json';
    const pwd = process.cwd();
    return {
        components,
        devBucket,
        prodBucket,
        manifestFile,
        devMode,
        pwd,
        gcsBucket,
    };
};
const friendifyError = (error) => `The value at: ${error.dataPath} is invalid. ${error.message}.`;
const unique = (ts) => [...new Set(ts)];
const throwIfErrors = (errors, fileType) => {
    const friendlyErrors = errors.map(friendifyError);
    const uniqueErrors = unique(friendlyErrors);
    if (uniqueErrors.length !== 0) {
        throw new Error(`Invalid ${fileType}: \n${JSON.stringify(uniqueErrors)}`);
    }
};
exports.validateManifestFile = (path) => {
    const fileExists = fs_1.existsSync(path);
    if (!fileExists) {
        throw new Error(`The file: \n${path}\n was not found.`);
    }
    const fileContents = fs_1.readFileSync(path, 'utf8');
    let parsedJson;
    try {
        parsedJson = JSON.parse(fileContents);
    }
    catch (e) {
        throw new Error(`The file:\n ${path}\n could not be parsed as JSON. `);
    }
    throwIfErrors(validate.validateManifest(parsedJson), 'manifest');
    return parsedJson;
};
exports.validateConfigFile = (path) => {
    const fileExists = fs_1.existsSync(path);
    if (!fileExists) {
        throw new Error(`The file: \n${path}\n was not found.`);
    }
    const fileContents = fs_1.readFileSync(path, 'utf8');
    let parsedJson;
    try {
        parsedJson = JSON.parse(fileContents);
    }
    catch (e) {
        throw new Error(`The file:\n ${path}\n could not be parsed as JSON. `);
    }
    throwIfErrors(validate.validateConfig(parsedJson), 'config');
    return true;
};
exports.getBuildableComponents = () => {
    const components = [];
    const lastComponentIdx = Object.keys(process.env)
        .filter((key) => key.startsWith('npm_package_dsccViz_components_'))
        .map((s) => s.replace('npm_package_dsccViz_components_', ''))
        .map((a) => parseInt(a, 10))
        .reduce((a, b) => (a > b ? a : b), 0);
    // Check for vizpack configuration
    for (let idx = 0; idx <= lastComponentIdx; idx++) {
        const jsonFile = process.env[`npm_package_dsccViz_components_${idx}_jsonFile`];
        if (!jsonFile) {
            throw util_1.invalidVizConfig(`components[${idx}].jsonFile`);
        }
        const cssFile = process.env[`npm_package_dsccViz_components_${idx}_cssFile`];
        // Require either jsFile or tsFile
        const jsFile = process.env[`npm_package_dsccViz_components_${idx}_jsFile`];
        const tsFile = process.env[`npm_package_dsccViz_components_${idx}_tsFile`];
        if (jsFile === undefined && tsFile === undefined) {
            throw util_1.invalidVizConfig(`components[${idx}].jsFile`);
        }
        components.push({
            jsonFile,
            cssFile,
            jsFile,
            tsFile,
        });
    }
    return components;
};
exports.getComponentIndex = (args, manifestPath) => {
    if (args.componentName) {
        const componentName = args.componentName;
        const manifest = exports.validateManifestFile(manifestPath);
        const idx = manifest.components.findIndex((component) => component.name === componentName);
        if (idx === -1) {
            throw new Error(`${componentName} is not present in your manifest.json`);
        }
        return idx.toString();
    }
    return '0';
};

[kmclaugh]

Link to breaking change commit

[niluparupasinghe]

By only updating the 'util.js' file I get this error.

modules\@google\dscc-scripts\build\viz\util.js:100 throw util_1.invalidVizConfig(components[${idx}].jsonFile); ^

Error: Your package.json must have a dsccViz.components[0].jsonFile entry: { "dsccViz": { "gcsDevBucket": "gs://validBucketPath", "gcsProdBucket": "gs://validBucketPath", "jsFile": "index.js", "jsonFile": "index.json", "cssFile": "index.css", "print": "printMessage.js" } } at invalidConfig (my_path\node_modules\@google\dscc-scripts\build\util.js:48:12) at Object.exports.invalidVizConfig (my_path\node_modules\@google\dscc-scripts\build\util.js:54:12) at exports.getBuildableComponents (my_path\node_modules\@google\dscc-scripts\build\viz\util.js:100 : 26) at Object. (my_path\webpack.config.js:7:20) at Module._compile (node:internal/modules/cjs/loader:1102:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1131:10) at Module.load (node:internal/modules/cjs/loader:967:32) at Function.Module._load (node:internal/modules/cjs/loader:807:14) at Module.require (node:internal/modules/cjs/loader:991:19) at require (node:internal/modules/cjs/helpers:92:18) at WEBPACK_OPTIONS (my_path\node_modules\webpack-cli\bin\utils\convert-argv.js:114:13) at requireConfig (my_path\node_modules\webpack-cli\bin\utils\convert-argv.js:116:6) at my_path\node_modules\webpack-cli\bin\utils\convert-argv.js:123:17 at Array.forEach () at module.exports (my_path\node_modules\webpack-cli\bin\utils\convert-argv.js:121:15) at Object. (my_path\node_modules\webpack-dev-server\bin\webpack-dev-server.js:84:40) Command failed: webpack-dev-server --open npm ERR! code 1 npm ERR! path my_path npm ERR! command failed npm ERR! command C:\Windows\system32\cmd.exe /d /s /c "dscc-scripts viz start"

npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\nilup\AppData\Local\npm-cache_logs\2020-11-14T16_47_04_950Z-debug.log

[mattkizaric-leveragelab]

@nilupaadl The way configuration files are written was changed. See here

[niluparupasinghe]

@nilupaadl The way configuration files are written was changed. See here

@mattkizaric-leveragelab Hi thanks for the direction, but does this mean that I have to do this manually for all the files? I mean what is the workaround to get "npm run start" to work? I'm bit new to the stuff.

[YannBrrd]

⬆️ on the thread. Is there a fix/workaround available?

[raddevon]

Thanks for the heads up. I ended up doing what you described and then copying the new util.js file generated by your suggested clone over the node_modeules/@google/dscc-scripts/build/viz/util.js file generated by the installation in the tutorial.

Anyway, the fix seems to be to update the `viz/util.js' file with the below code.

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.getComponentIndex = exports.getBuildableComponents = exports.validateConfigFile = exports.validateManifestFile = exports.validateBuildValues = void 0;
/**
 * @license
 * Copyright 2018 Google LLC
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
const validate = require("@google/dscc-validation");
const fs_1 = require("fs");
const args_1 = require("../args");
const util_1 = require("../util");
exports.validateBuildValues = (args) => {
    const components = exports.getBuildableComponents();
    const devBucket = process.env.npm_package_dsccViz_gcsDevBucket;
    if (devBucket === undefined) {
        throw util_1.invalidVizConfig('gcsDevBucket');
    }
    const prodBucket = process.env.npm_package_dsccViz_gcsProdBucket;
    if (prodBucket === undefined) {
        throw util_1.invalidVizConfig('gcsProdBucket');
    }
    const devMode = args.deployment === args_1.DeploymentChoices.PROD ? false : true;
    const gcsBucket = devMode ? devBucket : prodBucket;
    const manifestFile = 'manifest.json';
    const pwd = process.cwd();
    return {
        components,
        devBucket,
        prodBucket,
        manifestFile,
        devMode,
        pwd,
        gcsBucket,
    };
};
const friendifyError = (error) => `The value at: ${error.dataPath} is invalid. ${error.message}.`;
const unique = (ts) => [...new Set(ts)];
const throwIfErrors = (errors, fileType) => {
    const friendlyErrors = errors.map(friendifyError);
    const uniqueErrors = unique(friendlyErrors);
    if (uniqueErrors.length !== 0) {
        throw new Error(`Invalid ${fileType}: \n${JSON.stringify(uniqueErrors)}`);
    }
};
exports.validateManifestFile = (path) => {
    const fileExists = fs_1.existsSync(path);
    if (!fileExists) {
        throw new Error(`The file: \n${path}\n was not found.`);
    }
    const fileContents = fs_1.readFileSync(path, 'utf8');
    let parsedJson;
    try {
        parsedJson = JSON.parse(fileContents);
    }
    catch (e) {
        throw new Error(`The file:\n ${path}\n could not be parsed as JSON. `);
    }
    throwIfErrors(validate.validateManifest(parsedJson), 'manifest');
    return parsedJson;
};
exports.validateConfigFile = (path) => {
    const fileExists = fs_1.existsSync(path);
    if (!fileExists) {
        throw new Error(`The file: \n${path}\n was not found.`);
    }
    const fileContents = fs_1.readFileSync(path, 'utf8');
    let parsedJson;
    try {
        parsedJson = JSON.parse(fileContents);
    }
    catch (e) {
        throw new Error(`The file:\n ${path}\n could not be parsed as JSON. `);
    }
    throwIfErrors(validate.validateConfig(parsedJson), 'config');
    return true;
};
exports.getBuildableComponents = () => {
    const components = [];
    const lastComponentIdx = Object.keys(process.env)
        .filter((key) => key.startsWith('npm_package_dsccViz_components_'))
        .map((s) => s.replace('npm_package_dsccViz_components_', ''))
        .map((a) => parseInt(a, 10))
        .reduce((a, b) => (a > b ? a : b), 0);
    // Check for vizpack configuration
    for (let idx = 0; idx <= lastComponentIdx; idx++) {
        const jsonFile = process.env[`npm_package_dsccViz_components_${idx}_jsonFile`];
        if (!jsonFile) {
            throw util_1.invalidVizConfig(`components[${idx}].jsonFile`);
        }
        const cssFile = process.env[`npm_package_dsccViz_components_${idx}_cssFile`];
        // Require either jsFile or tsFile
        const jsFile = process.env[`npm_package_dsccViz_components_${idx}_jsFile`];
        const tsFile = process.env[`npm_package_dsccViz_components_${idx}_tsFile`];
        if (jsFile === undefined && tsFile === undefined) {
            throw util_1.invalidVizConfig(`components[${idx}].jsFile`);
        }
        components.push({
            jsonFile,
            cssFile,
            jsFile,
            tsFile,
        });
    }
    return components;
};
exports.getComponentIndex = (args, manifestPath) => {
    if (args.componentName) {
        const componentName = args.componentName;
        const manifest = exports.validateManifestFile(manifestPath);
        const idx = manifest.components.findIndex((component) => component.name === componentName);
        if (idx === -1) {
            throw new Error(`${componentName} is not present in your manifest.json`);
        }
        return idx.toString();
    }
    return '0';
};

I'm also having this issue. Copying this code to node_modeules/@google/dscc-scripts/build/viz/util.js worked for me to fix npm run start, but it broke update_message. I was getting The "path" argument must be of type string. Received type undefined whenever I ran npm run update_message.

I discovered the utils fix along with an apparently outdated package.json schema in the template were causing the new issue. By reverting node_modeules/@google/dscc-scripts/build/viz/util.js, I got a new error when running npm run update_message:

Your package.json must have a dsccViz.jsonFile entry:
{
  "dsccViz": {
    "gcsDevBucket": "gs://validBucketPath",
    "gcsProdBucket": "gs://validBucketPath",
    "jsFile": "index.js",
    "jsonFile": "index.json",
    "cssFile": "index.css",
    "print": "printMessage.js"
  }
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ update_message: `dscc-scripts viz update_message -f object`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ update_message script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I found my package.json included a dsccViz key, but the missing jsonFile key was nested one level deeper in a components array containing a single object. Here's what that looked like:

{
  "dsccViz": {
    "gcsDevBucket": "gs://dm-pop-up-list-visualization/dev",
    "gcsProdBucket": "gs://dm-pop-up-list-visualization/prod",
    "components": [
      {
        "jsFile": "index.js",
        "jsonFile": "index.json",
        "cssFile": "index.css"
      }
    ]
  },
  "scripts": {
    "build:dev": "dscc-scripts viz build -d dev",
    "build:prod": "dscc-scripts viz build -d prod",
    "push:dev": "dscc-scripts viz push -d dev",
    "push:prod": "dscc-scripts viz push -d prod",
    "update_message": "dscc-scripts viz update_message -f object",
    "start": "dscc-scripts viz start"
  },
  "devDependencies": {
    "@google/dscc": "^0.3.17",
    "@google/dscc-scripts": "^1.0.17",
    "copy-webpack-plugin": "^4.6.0"
  }
}

I changed it to this instead:

{
  "dsccViz": {
    "gcsDevBucket": "gs://dm-pop-up-list-visualization/dev",
    "gcsProdBucket": "gs://dm-pop-up-list-visualization/prod",
    "jsFile": "index.js",
    "jsonFile": "index.json",
    "cssFile": "index.css"
  },
  "scripts": {
    "build:dev": "dscc-scripts viz build -d dev",
    "build:prod": "dscc-scripts viz build -d prod",
    "push:dev": "dscc-scripts viz push -d dev",
    "push:prod": "dscc-scripts viz push -d prod",
    "update_message": "dscc-scripts viz update_message -f object",
    "start": "dscc-scripts viz start"
  },
  "devDependencies": {
    "@google/dscc": "^0.3.17",
    "@google/dscc-scripts": "^1.0.17",
    "copy-webpack-plugin": "^4.6.0"
  }
}

and npm run update_message now works. This breaks npm run start again.

[EonYang]

Workaround: use dscc-gen@2.0.30

Since the issue was introduced with the latest release2.0.31, we can simply go back and use the previous version until they fix it.

# create a new project with the previous version of `dscc-gen`
npx @google/dscc-gen@2.0.30 viz

Then move your src/ folder to the new project manually.

[monishwhisper]

dscc-gen@2.0.30

Why not just use the version listed instead of also moving the files around? I got around by just using this version number suggested by @EonYang

[EonYang]

dscc-gen@2.0.30

Why not just use the version listed instead of also moving the files around? I got around by just using this version number suggested by @EonYang

You're right. I was worried about dependency conflicts but seems like yarn and npm are smart enough to resolve it nicely.

[danielovalle]

I used the dscc-gen@2.0.30 `➜ test2 npm run start

start dscc-scripts viz start

node:internal/validators:129 throw new ERR_INVALID_ARG_TYPE(name, 'string', value); ^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined at new NodeError (node:internal/errors:329:5) at validateString (node:internal/validators:129:11) at Object.join (node:path:1081:7) at Object. (/Users/danielovalle/github/dovalle-viz/test2/webpack.config.js:6:26) at Module._compile (node:internal/modules/cjs/loader:1108:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1137:10) at Module.load (node:internal/modules/cjs/loader:973:32) at Function.Module._load (node:internal/modules/cjs/loader:813:14) at Module.require (node:internal/modules/cjs/loader:997:19) at require (node:internal/modules/cjs/helpers:92:18) at WEBPACK_OPTIONS (/Users/danielovalle/github/dovalle-viz/test2/node_modules/webpack-cli/bin/utils/convert-argv.js:114:13) at requireConfig (/Users/danielovalle/github/dovalle-viz/test2/node_modules/webpack-cli/bin/utils/convert-argv.js:116:6) at /Users/danielovalle/github/dovalle-viz/test2/node_modules/webpack-cli/bin/utils/convert-argv.js:123:17 at Array.forEach () at module.exports (/Users/danielovalle/github/dovalle-viz/test2/node_modules/webpack-cli/bin/utils/convert-argv.js:121:15) at Object. (/Users/danielovalle/github/dovalle-viz/test2/node_modules/webpack-dev-server/bin/webpack-dev-server.js:84:40) { code: 'ERR_INVALID_ARG_TYPE' } Command failed: webpack-dev-server --open`

[taljuk01]

I tried all the ideas in this post and none worked. Did someone solve it?

[taljuk01]

I found a solution. First of all install the latest version of npx: npx @google/dscc-gen viz

Then if you go to node_modules/@google/dscc-scripts/build/viz/util.js, replace the code for this one here.

This will solve the first problem: TypeError: getBuildableComponents is not a function

Now you will get this error:

Your package.json must have a dsccViz.jsonFile entry:
{
  "dsccViz": {
    "gcsDevBucket": "gs://validBucketPath",
    "gcsProdBucket": "gs://validBucketPath",
    "jsFile": "index.js",
    "jsonFile": "index.json",
    "cssFile": "index.css",
    "print": "printMessage.js"
  }
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ update_message: `dscc-scripts viz update_message -f object`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ update_message script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

To solve this error go to node_modules/@google/dscc-scripts/build/viz/util.js and replace this lines:

Line 97 to 115:

 // Check for vizpack configuration
    for (let idx = 0; idx <= lastComponentIdx; idx++) {
        const jsonFile = "index.json"//process.env[`npm_package_dsccViz_components_${idx}_jsonFile`];
        /*if (!jsonFile) {
            throw util_1.invalidVizConfig(`components[${idx}].jsonFile`);
        }*/
        const cssFile = "index.css"//process.env[`npm_package_dsccViz_components_${idx}_cssFile`];
        // Require either jsFile or tsFile
        const jsFile = "index.js"//process.env[`npm_package_dsccViz_components_${idx}_jsFile`];
        const tsFile = process.env[`npm_package_dsccViz_components_${idx}_tsFile`];
        if (jsFile === undefined && tsFile === undefined) {
            throw util_1.invalidVizConfig(`components[${idx}].jsFile`);
        }
        components.push({
            jsonFile,
            cssFile,
            jsFile,
            tsFile,
        });
    }

The problem was with process.env. It's not finding the virtual environment variable npm_package_dsccViz_components_${idx}_jsonFile

[sesi]

I was able to get a workaround to run all dscc-gen scripts (start, update_message, build, push) with a few extra steps since just using the previous version (dscc-gen@2.0.30) is no longer a viable solution as @danielovalle noted above.

First, follow the steps by @taljuk01 in the post above:

1. Create a viz project with the latest version by running npx @google/dscc-gen viz
2. Use the code provided by @kmclaugh here to replace thenode_modules/@google/dscc-scripts/build/viz/util.js file
3. Replace lines 97-115 with code here

This will get npm run start working (yay!) but npm run build, npm run push, and npm run update_message will still need fixing.

If you attempt to run update_message or build you might get an error like this:

Your package.json must have a dsccViz.gcsDevBucket entry

In node_modules/@google/dscc-scripts/build/viz/util.js lines 26-33 it is failing to pick up the environment variables

    const devBucket = process.env.npm_package_dsccViz_gcsDevBucket;
    if (devBucket === undefined) {
        throw util_1.invalidVizConfig('gcsDevBucket');
    }
    const prodBucket = process.env.npm_package_dsccViz_gcsProdBucket;
    if (prodBucket === undefined) {
        throw util_1.invalidVizConfig('gcsProdBucket');
    }

Define your dev and prod buckets instead of using the environment variables

    const devBucket = "gs://please-help-me/my-path-dev"; //process.env.npm_package_dsccViz_gcsDevBucket;
    if (devBucket === undefined) {
        throw util_1.invalidVizConfig('gcsDevBucket');
    }
    const prodBucket = "gs://please-help-me/my-path-prod"; //process.env.npm_package_dsccViz_gcsProdBucket;
    if (prodBucket === undefined) {
        throw util_1.invalidVizConfig('gcsProdBucket');
    }

Now you might also encounter this error if you try to run build or update_message:

The "path" argument must be of type string. Received type undefined

In node_modules/@google/dscc-scripts/build/viz/build.js, some of the buildValues (jsonFile, cssFile, jsFile, tsFile) are now nested inside the components. For instance, "index.json" is now buildValues.components[0].jsonFile not buildValues.jsonFile which is undefined. Update these values so they are no longer undefined.

This can just be done below line 24 at the start of the buildOptions function

const buildOptions = (buildValues) => {
    const plugins = [

Reassign them with their values directly (ex. buildValues.jsonFile = "index.json") or by accessing the values from components as shown below

const buildOptions = (buildValues) => {
    buildValues.jsonFile = buildValues.components[0].jsonFile; 
    buildValues.cssFile = buildValues.components[0].cssFile;
    buildValues.jsFile = buildValues.components[0].jsFile;
    buildValues.tsFile = buildValues.components[0].tsFile; 
    const plugins = [

Now we can do the same in node_modules/@google/dscc-scripts/build/viz/message.js where the buildOptions function begins in lines 26-27

const buildOptions = (buildValues, args) => {
    let transformString;

Reassign them the same way you did previously in build.js

const buildOptions = (buildValues, args) => {
    buildValues.jsonFile = buildValues.components[0].jsonFile; 
    buildValues.cssFile = buildValues.components[0].cssFile;
    buildValues.jsFile = buildValues.components[0].jsFile;
    buildValues.tsFile = buildValues.components[0].tsFile; 
    let transformString;

Now all of the commands for dscc-gen should work! May be a short term option until a fix is implemented.

[monishwhisper]

Can't believe this is not receiving enough attention. The work arounds seem like so much work!

[ibraelillo]

Hi guys, I've just took the function code which is present here in the comments, and put it directly inside my webpack.config.js and it worked. No need to modify node_modules as this is not the right way to do it if you think about deploys in CD/CI or you re-install with clean.

[joshuasprow]

Ditto @ibraelillo

Double ditto @monishwhisper !!

I hope there aren't too many people having this problem. It's pretty discouraging when you're trying to get work done and a first-party library is busted.

Also, there are people copy/pasting into node_modules... that's bad, friends.

Here's some more copypasta for someone looking for something small and useful for local development. It doesn't fix the update_message, build, or push scripts, though.

[ line 5 ]
// Get outta here!!
// const {getBuildableComponents} = require('@google/dscc-scripts/build/viz/util');

const getBuildableComponents = () => {
  const packageJson = JSON.parse(fs.readFileSync("package.json", "utf-8"));

  if (!("dsccViz" in packageJson)) {
    throw new Error(`missing "dsccViz" in package.json`);
  }
  if (!("components" in packageJson.dsccViz)) {
    throw new Error(`missing "dsccViz.components" in package.json`);
  }

  const { components } = packageJson.dsccViz;

  if (components.length === 0) {
    throw new Error("no components to build");
  }
  if (typeof components[0].jsFile != "string") {
    throw new Error("no components.jsFile to build");
  }

  return components;
};

[taljuk01mh]

There is one more bug to resolve in node_modules/@google/dscc-scripts/build/viz/build.js Line 124 needs to be commented out, because sometimes it's fail to validate the index.json file:

// util.validateConfigFile(configDest);  <------- COMMENT THIS LINE
const manifestDest = path.resolve(cwd, 'build', buildValues.manifestFile);
util.validateManifestFile(manifestDest);

};

[hieumdd]

Hi guys, I've just took the function code which is present here in the comments, and put it directly inside my webpack.config.js and it worked. No need to modify node_modules as this is not the right way to do it if you think about deploys in CD/CI or you re-install with clean.

Can you share how did you do it for webpack.config.js?

[Nomane]

That's just incredible, hope this issue will be fix soon

[jtrobec]

From what I can tell, the code in the repo is OK, it's just that someone needs to publish a new version of dscc-scripts to npm. I was able to work around this issue by (these are from memory, may need to fiddle with this):

1) clone this repo locally 2) cd <repo>/packages/dscc-scripts 3) yarn 4) yarn build 5) npm install 6) npm link 7) At this point, follow the basic instructions for running npx @google/dscc-components viz to set up the project. 8) Change into the project directory, and do npm run start this should fail with the error at the beginning of this thread. 9) After the error, run npm link @google/dscc-scripts. 10) Now run npm run start again, it should start.

As far as I can tell, all this did was replace the dscc-scripts in my node_modules (v1.0.17) with the latest code from github. So I think we just need a new version published to npm to resolve this issue.

@diminishedprime sorry if you're not the right person to bug for this, but does this sound right?

[pixelthing]

TypeError: getBuildableComponents is not a function

Adding my +1 to this - It's now mid Sept & I've tried a couple of suggested solutions without luck, but there are now so many solutions put forward above (not all of them work for most people), it's hard to know how much time to spend trashing my set-up, or just give up the idea of creating a community data viz completely.

It's a little bit demoralising. Is there any chance updated tooling is coming sometime soon?

[gowtham1337]

@diminishedprime Could you take a look at this issue, please?

[giordanocolombi]

is there any possibility that this bug will be fixed soon? I would really like using this feature

[pixelthing]

This seems crazy - has no one been able to create community visualizations since Nov 2020 (a year ago at this point)? There are other issues against this github repo from 2021, so how are those people getting the tools to run?

There must be some other way of building and testing community viz if these tools don't run? Most of the above cut/pastes and hacks around no longer work (eg, Webpack seems to have changed it's import syntax) or are half documented. I'm running a 64bit Intel Mac. Is it possible it's a Mac only problem and this runs on Linux?

Either I'm missing something or I'm taking crazy pills. Anyone know of an alternative route to getting community viz tools working?

Help! (apologies for the direct callout) @diminishedprime @anweshan @y3l2n

[anweshan]

@pixelthing It looks like the instructions using dscc-gen are what you are having issues with, the more common way of getting community viz tools to run is to use the dscc library directly following the instructions here: https://www.npmjs.com/package/@google/dscc

[Kixell-NicolasJardillier]

In fact using :

node v12.20.0
npm 6.14.8
npx @google/dscc-gen@2.0.30 viz

works !

If you use all in latest version, it fails

@diminishedprime or @anweshan : could we have the correct environment configuration to use at least the last version of dscc-gen correctly please ?

[aschor]

well, almost 2 years, still broken. nice.

tried to read the code : there are many references to environment variables, but NOTHING declares those variables (except in the test code). That's absurd coding (since there is a conf file for not having environment variables), absurd documentation ... etc.

I simply can't get this thing to work, even with all intel here.

edit : if anyone is interested, I managed to get things to work OK, but maybe not in a production-level-quality (I discovered npm here. And hated it right away when trying unsuccessfully to use the link/install commands).

here is my local fix and my local modified usage readme

[jamesyoon11]

Looks like they fixed issue here. However, it was after 1.0.17 which is the latest version that they published to npmjs.

https://github.com/googledatastudio/tooling/commits/main/packages/dscc-scripts

https://www.npmjs.com/package/@google/dscc-scripts

So, I just created my own repo and it works with updating package.json as.

{
  "dsccViz": {
    "gcsDevBucket": "",
    "gcsProdBucket": "",
    "components": [
      {
        "jsFile": "index.js",
        "jsonFile": "index.json",
        "cssFile": "index.css"
      }
    ]
  },
  "scripts": {
    "build:dev": "dscc-scripts viz build -d dev",
    "build:prod": "dscc-scripts viz build -d prod",
    "push:dev": "dscc-scripts viz push -d dev",
    "push:prod": "dscc-scripts viz push -d prod",
    "update_message": "dscc-scripts viz update_message -f object",
    "start": "dscc-scripts viz start"
  },
  "devDependencies": {
    "@google/dscc": "^0.3.17",
    "@google/dscc-scripts": "https://github.com/jamesyoon11/dscc-scripts.git",
    "copy-webpack-plugin": "^4.6.0"
  }
}

[eplusx]

Surprised that this has been open for years...

Hoping this should be fixed, I filed a bug at https://issuetracker.google.com/issues/301488776. I don't expect a quick resolution as this tool looks heavily deprioritized... :(

[eplusx]

OK, it seems I found a workaround for live editing that covers new errors not mentioned above.

1. Install dscc-gen as usual.

   npx @google/dscc-gen viz

   Make sure the dev and prod directories are in a GCS bucket with fine-grained ACL. (Issue Tracker, Docs, this is needed because dscc-gen runs a legacy gsutil acl get ....) You can change it in your bucket's CONFIGURATION -> Permissions -> Access control.

2. Change directory into your project, and a. Make a fix posted by joshuasprow. b. Or alternatively, you could just mimic the component object:

     let component = {
       jsFile: 'index.js',
       cssFile: 'index.css'
     };

3. Make a change at the end of node_modules/webpack/lib/util/createHash.js to replace createHash("md4") with createHash("sha1"). Your switch statement will look like this:

   switch (algorithm) {
       // TODO add non-cryptographic algorithm here
       case "debug":
           return new DebugHash();
       case "md4":
           return new BulkUpdateDecorator(require("crypto").createHash("sha1"));
       default:
           return new BulkUpdateDecorator(require("crypto").createHash(algorithm));
   }

   This will throw away ERR_OSSL_EVP_UNSUPPORTED error. (See also this article.)

I know this is nasty -- the last change cannot be tracked by Git. :( That said there are a lot of createHash("md4") calls in webpack@4.41.2 (which is installed by copy-webpack-plugin@4.6.0, published in 2018) and you'd see dependency conflicts with @google/dscc if you try to use new one.

A cleaner solution would be updating those versions in the original reporitories (which I don't expect to occur sometime soon).

[kevin-vaghasiya]

Solution by @eplusx works very well for running locally.

If you want to build & deploy follow this steps. go to node_modules/@google/dscc-scripts/build/viz/util.js

replace following environment variables with your file names

process.env.npm_package_dsccViz_cssFile => index.css
process.env.npm_package_dsccViz_jsonFile => index.json
process.env.npm_package_dsccViz_jsFile => index.js
process.env.npm_package_dsccViz_gcsDevBucket => DEV_BUCKET_PATH
process.env.npm_package_dsccViz_gcsProdBucket => PROD_BUCKET_PATH 

[harri35]

Thanks, guys, for all the instructions! I would have been pretty much stuck without it. I just needed to make a small change to our visualization, didn't expect the setup to be so hands-on :D Hopefully, this will be fixed sometime in the near future.