search

evanw / esbuild

An extremely fast bundler for the web
https://esbuild.github.io/
MIT License
38.26k stars 1.16k forks source link

"Cannot find module" error with What3Words library when bundling TypeScript with platform=node #1753

Closed jamescrowley closed 3 years ago

jamescrowley commented 3 years ago

I'm not clear if this is an issue with the What3Words node library, or the dependency, but when bundling with platform=node, the output JS does not include all the required dependencies.

I noticed in the documentation the limitation around dynamic dependencies (https://esbuild.github.io/api/#non-analyzable-imports) however, the imports don't appear to be dynamic in the original source code https://github.com/what3words/w3w-node-wrapper/blob/master/src/index.ts#L1 or in the transpiled output published to NPM.

Can anyone clarify where the issue lies?

I'm running esbuild 0.13.12, and node v12.17.0

index.ts:

import * as what3words from "@what3words/api";

exports.handler = async function (event) {
  console.log('Received S3 event:', JSON.stringify(event, null, 2));
  what3words.setOptions({ key: "abc" });
  const what3wordsResponse = await what3words.convertTo3wa({lat:-23, lng:2});
  console.log(`Got what3words ${JSON.stringify(what3wordsResponse)}`);
};

package.json:

{
  ...
  "dependencies": {
    "@what3words/api": "^3.3.6"
  }
}

This is the output when running esbuild --bundle "index.ts"

(click to expand output)
```js (() => { var __esm = (fn, res) => function __init() { return fn && (res = (0, fn[Object.keys(fn)[0]])(fn = 0)), res; }; var __commonJS = (cb, mod) => function __require() { return mod || (0, cb[Object.keys(cb)[0]])((mod = { exports: {} }).exports, mod), mod.exports; }; // node_modules/@what3words/api/es2015/constants.js var init_constants = __esm({ "node_modules/@what3words/api/es2015/constants.js"() { } }); // node_modules/@what3words/api/es2015/utils.js var GLOBAL_OPTIONS, searchParams, coordinatesToString, setOptions; var init_utils = __esm({ "node_modules/@what3words/api/es2015/utils.js"() { init_constants(); GLOBAL_OPTIONS = { key: "", baseUrl: "https://api.what3words.com/v3" }; searchParams = (data) => Object.keys(data).map((key) => `${encodeURIComponent(key)}=${encodeURIComponent(data[key])}`).join("&"); coordinatesToString = (coordinates) => `${coordinates.lat},${coordinates.lng}`; setOptions = (options) => { GLOBAL_OPTIONS = Object.assign(Object.assign({}, GLOBAL_OPTIONS), options); }; } }); // node_modules/@what3words/api/es2015/version.js var version; var init_version = __esm({ "node_modules/@what3words/api/es2015/version.js"() { version = "3.3.6"; } }); // node_modules/@what3words/api/es2015/fetch.js var fetchGet; var init_fetch = __esm({ "node_modules/@what3words/api/es2015/fetch.js"() { init_utils(); init_version(); fetchGet = (url, data = {}, signal) => { const options = { method: "GET", headers: { "X-W3W-Wrapper": `what3words-JavaScript/${version} (${navigator.userAgent})` } }; if (signal !== void 0) { options.signal = signal; } if (typeof GLOBAL_OPTIONS.key === "string" && GLOBAL_OPTIONS.key.length > 0) { data["key"] = GLOBAL_OPTIONS.key; } if (GLOBAL_OPTIONS.headers) { options.headers = Object.assign(Object.assign({}, options.headers), GLOBAL_OPTIONS.headers); } let hasError = false; return fetch(`${GLOBAL_OPTIONS.baseUrl}/${url}?${searchParams(data)}`, options).then((response) => { hasError = !response.ok; const contentType = response.headers.get("content-type"); if (!contentType) return null; if (contentType.indexOf("application/json") !== -1) return response.json(); return response.text(); }).then((data2) => { if (hasError) { throw data2.error; } return data2; }); }; } }); // node_modules/@what3words/api/es2015/requests/autosuggest.js var init_autosuggest = __esm({ "node_modules/@what3words/api/es2015/requests/autosuggest.js"() { init_utils(); init_fetch(); } }); // node_modules/@what3words/api/es2015/requests/autosuggest-selection.js var init_autosuggest_selection = __esm({ "node_modules/@what3words/api/es2015/requests/autosuggest-selection.js"() { init_fetch(); init_utils(); init_autosuggest(); } }); // node_modules/@what3words/api/es2015/requests/available-languages.js var init_available_languages = __esm({ "node_modules/@what3words/api/es2015/requests/available-languages.js"() { init_fetch(); } }); // node_modules/@what3words/api/es2015/requests/convert-to-3wa.js var convertTo3waBase, convertTo3wa; var init_convert_to_3wa = __esm({ "node_modules/@what3words/api/es2015/requests/convert-to-3wa.js"() { init_utils(); init_fetch(); convertTo3waBase = (coordinates, language, format, signal) => { const requestOptions = { coordinates: coordinatesToString(coordinates) }; if (language !== void 0) { requestOptions["language"] = language; } if (format !== void 0) { requestOptions["format"] = format; } return fetchGet("convert-to-3wa", requestOptions, signal); }; convertTo3wa = (coordinates, language, signal) => convertTo3waBase(coordinates, language, "json", signal); } }); // node_modules/@what3words/api/es2015/requests/convert-to-coordinates.js var init_convert_to_coordinates = __esm({ "node_modules/@what3words/api/es2015/requests/convert-to-coordinates.js"() { init_fetch(); } }); // node_modules/@what3words/api/es2015/requests/grid-section.js var init_grid_section = __esm({ "node_modules/@what3words/api/es2015/requests/grid-section.js"() { init_utils(); init_fetch(); } }); // node_modules/@what3words/api/es2015/index.js var init_es2015 = __esm({ "node_modules/@what3words/api/es2015/index.js"() { init_constants(); init_autosuggest(); init_autosuggest_selection(); init_available_languages(); init_convert_to_3wa(); init_convert_to_coordinates(); init_grid_section(); init_utils(); } }); // index.ts var require_s3_test = __commonJS({ "index.ts"(exports) { init_es2015(); exports.handler = async function(event) { console.log("Received S3 event:", JSON.stringify(event, null, 2)); setOptions({ key: "abc" }); const what3wordsResponse = await convertTo3wa({ lat: -23, lng: 2 }); console.log(`Got what3words ${JSON.stringify(what3wordsResponse)}`); }; } }); require_s3_test(); })(); ```

This is the output when running esbuild --bundle "index.ts" --platform=node. This appears to be much shorter and does not include the required files. When running it fails with "Cannot find module './constants'"

(click to expand output)
```js var __create = Object.create; var __defProp = Object.defineProperty; var __getOwnPropDesc = Object.getOwnPropertyDescriptor; var __getOwnPropNames = Object.getOwnPropertyNames; var __getProtoOf = Object.getPrototypeOf; var __hasOwnProp = Object.prototype.hasOwnProperty; var __markAsModule = (target) => __defProp(target, "__esModule", { value: true }); var __commonJS = (cb, mod) => function __require() { return mod || (0, cb[Object.keys(cb)[0]])((mod = { exports: {} }).exports, mod), mod.exports; }; var __reExport = (target, module2, desc) => { if (module2 && typeof module2 === "object" || typeof module2 === "function") { for (let key of __getOwnPropNames(module2)) if (!__hasOwnProp.call(target, key) && key !== "default") __defProp(target, key, { get: () => module2[key], enumerable: !(desc = __getOwnPropDesc(module2, key)) || desc.enumerable }); } return target; }; var __toModule = (module2) => { return __reExport(__markAsModule(__defProp(module2 != null ? __create(__getProtoOf(module2)) : {}, "default", module2 && module2.__esModule && "default" in module2 ? { get: () => module2.default, enumerable: true } : { value: module2, enumerable: true })), module2); }; // node_modules/@what3words/api/umd/index.js var require_umd = __commonJS({ "node_modules/@what3words/api/umd/index.js"(exports2, module2) { (function(factory) { if (typeof module2 === "object" && typeof module2.exports === "object") { var v = factory(require, exports2); if (v !== void 0) module2.exports = v; } else if (typeof define === "function" && define.amd) { define(["require", "exports", "./constants", "./requests/autosuggest", "./requests/autosuggest-selection", "./requests/available-languages", "./requests/convert-to-3wa", "./requests/convert-to-coordinates", "./requests/grid-section", "./utils"], factory); } })(function(require2, exports3) { "use strict"; Object.defineProperty(exports3, "__esModule", { value: true }); exports3.valid3wa = exports3.getWords = exports3.getOptions = exports3.setOptions = exports3.gridSectionGeoJson = exports3.gridSection = exports3.convertToCoordinatesGeoJson = exports3.convertToCoordinates = exports3.convertTo3waGeoJson = exports3.convertTo3wa = exports3.availableLanguages = exports3.autosuggestSelection = exports3.autosuggest = exports3.W3W_REGEX = void 0; var constants_1 = require2("./constants"); Object.defineProperty(exports3, "W3W_REGEX", { enumerable: true, get: function() { return constants_1.W3W_REGEX; } }); var autosuggest_1 = require2("./requests/autosuggest"); Object.defineProperty(exports3, "autosuggest", { enumerable: true, get: function() { return autosuggest_1.autosuggest; } }); var autosuggest_selection_1 = require2("./requests/autosuggest-selection"); Object.defineProperty(exports3, "autosuggestSelection", { enumerable: true, get: function() { return autosuggest_selection_1.autosuggestSelection; } }); var available_languages_1 = require2("./requests/available-languages"); Object.defineProperty(exports3, "availableLanguages", { enumerable: true, get: function() { return available_languages_1.availableLanguages; } }); var convert_to_3wa_1 = require2("./requests/convert-to-3wa"); Object.defineProperty(exports3, "convertTo3wa", { enumerable: true, get: function() { return convert_to_3wa_1.convertTo3wa; } }); Object.defineProperty(exports3, "convertTo3waGeoJson", { enumerable: true, get: function() { return convert_to_3wa_1.convertTo3waGeoJson; } }); var convert_to_coordinates_1 = require2("./requests/convert-to-coordinates"); Object.defineProperty(exports3, "convertToCoordinates", { enumerable: true, get: function() { return convert_to_coordinates_1.convertToCoordinates; } }); Object.defineProperty(exports3, "convertToCoordinatesGeoJson", { enumerable: true, get: function() { return convert_to_coordinates_1.convertToCoordinatesGeoJson; } }); var grid_section_1 = require2("./requests/grid-section"); Object.defineProperty(exports3, "gridSection", { enumerable: true, get: function() { return grid_section_1.gridSection; } }); Object.defineProperty(exports3, "gridSectionGeoJson", { enumerable: true, get: function() { return grid_section_1.gridSectionGeoJson; } }); var utils_1 = require2("./utils"); Object.defineProperty(exports3, "setOptions", { enumerable: true, get: function() { return utils_1.setOptions; } }); Object.defineProperty(exports3, "getOptions", { enumerable: true, get: function() { return utils_1.getOptions; } }); Object.defineProperty(exports3, "getWords", { enumerable: true, get: function() { return utils_1.getWords; } }); Object.defineProperty(exports3, "valid3wa", { enumerable: true, get: function() { return utils_1.valid3wa; } }); }); } }); // index.ts var what3words = __toModule(require_umd()); exports.handler = async function(event) { console.log("Received S3 event:", JSON.stringify(event, null, 2)); what3words.setOptions({ key: "abc" }); const what3wordsResponse = await what3words.convertTo3wa({ lat: -23, lng: 2 }); console.log(`Got what3words ${JSON.stringify(what3wordsResponse)}`); }; ```

I've raised a ticket with what3words here: https://github.com/what3words/w3w-node-wrapper/issues/30 for reference

evanw commented 3 years ago

the imports don't appear to be dynamic ... in the transpiled output published to NPM

They are:

(function (factory) {
    if (typeof module === "object" && typeof module.exports === "object") {
        var v = factory(require, exports);
        if (v !== undefined) module.exports = v;
    }
    else if (typeof define === "function" && define.amd) {
        define(["require", "exports", "./constants", "./requests/autosuggest", "./requests/autosuggest-selection", "./requests/available-languages", "./requests/convert-to-3wa", "./requests/convert-to-coordinates", "./requests/grid-section", "./utils"], factory);
    }
})(function (require, exports) {
    "use strict";
    Object.defineProperty(exports, "__esModule", { value: true });
    exports.valid3wa = exports.getWords = exports.getOptions = exports.setOptions = exports.gridSectionGeoJson = exports.gridSection = exports.convertToCoordinatesGeoJson = exports.convertToCoordinates = exports.convertTo3waGeoJson = exports.convertTo3wa = exports.availableLanguages = exports.autosuggestSelection = exports.autosuggest = exports.W3W_REGEX = void 0;
    var constants_1 = require("./constants");
    Object.defineProperty(exports, "W3W_REGEX", { enumerable: true, get: function () { return constants_1.W3W_REGEX; } });
    var autosuggest_1 = require("./requests/autosuggest");
    Object.defineProperty(exports, "autosuggest", { enumerable: true, get: function () { return autosuggest_1.autosuggest; } });
    var autosuggest_selection_1 = require("./requests/autosuggest-selection");
    Object.defineProperty(exports, "autosuggestSelection", { enumerable: true, get: function () { return autosuggest_selection_1.autosuggestSelection; } });
    var available_languages_1 = require("./requests/available-languages");
    Object.defineProperty(exports, "availableLanguages", { enumerable: true, get: function () { return available_languages_1.availableLanguages; } });
    var convert_to_3wa_1 = require("./requests/convert-to-3wa");
    Object.defineProperty(exports, "convertTo3wa", { enumerable: true, get: function () { return convert_to_3wa_1.convertTo3wa; } });
    Object.defineProperty(exports, "convertTo3waGeoJson", { enumerable: true, get: function () { return convert_to_3wa_1.convertTo3waGeoJson; } });
    var convert_to_coordinates_1 = require("./requests/convert-to-coordinates");
    Object.defineProperty(exports, "convertToCoordinates", { enumerable: true, get: function () { return convert_to_coordinates_1.convertToCoordinates; } });
    Object.defineProperty(exports, "convertToCoordinatesGeoJson", { enumerable: true, get: function () { return convert_to_coordinates_1.convertToCoordinatesGeoJson; } });
    var grid_section_1 = require("./requests/grid-section");
    Object.defineProperty(exports, "gridSection", { enumerable: true, get: function () { return grid_section_1.gridSection; } });
    Object.defineProperty(exports, "gridSectionGeoJson", { enumerable: true, get: function () { return grid_section_1.gridSectionGeoJson; } });
    var utils_1 = require("./utils");
    Object.defineProperty(exports, "setOptions", { enumerable: true, get: function () { return utils_1.setOptions; } });
    Object.defineProperty(exports, "getOptions", { enumerable: true, get: function () { return utils_1.getOptions; } });
    Object.defineProperty(exports, "getWords", { enumerable: true, get: function () { return utils_1.getWords; } });
    Object.defineProperty(exports, "valid3wa", { enumerable: true, get: function () { return utils_1.valid3wa; } });
});
//# sourceMappingURL=index.js.map

Content from here: https://unpkg.com/@what3words/api@3.3.6/umd/index.js. The problem is the package you are trying to use doesn't work with bundlers that require statically-analyzable CommonJS modules (notice how require is stored to a variable and passed into another function before being used). This only happens for node because if you use --platform=node then the main field in package.json is used, and if you use --platform=browser then the module field in package.json is used instead so it loads a different file.

You could try to work around this with --platform=node --main-fields=module,main which will always prefer module over main, but while that may fix this package it may also break other packages in certain cases (example: #363). See also: https://esbuild.github.io/api/#main-fields. You can also write a simple plugin to intercept @what3words/api and redirect it to @what3words/api/es2015/index.js instead. See also: https://esbuild.github.io/plugins/#resolve-callbacks.

jamescrowley commented 3 years ago

Thanks for the detailed response @evanw, much appreciated. Unfortunately the es2015 in the library references navigator which creates its own problems.

Do you happen to know what is required for the transpilation on the library side to make this 'friendly' for the static analysis esbuild relies on, so I could propose a patch? The library source is pretty straightforward, and the underlying typescript all uses static imports so I'm assuming it's something to do with their babel typescript/transpilation config? I've tried googling around but not found the right guidance yet.

evanw commented 3 years ago

Here's a simple patch for that package:

--- a/package.json
+++ b/package.json
@@ -30,7 +30,7 @@
     "clear": "rm -rf dist",
     "create-version": "node ./scripts/version.js",
     "build-js": "tsc --target es2015 --module es2015 --outDir dist/es2015",
-    "build-node": "tsc --target es5 --module umd --outDir dist/umd",
+    "build-node": "tsc --target es5 --module CommonJS --outDir dist/umd",
     "build-script": "node ./scripts/index.js",
     "publish": "npm publish dist",
     "test": "jest",

See https://www.typescriptlang.org/tsconfig#module for more information about TypeScript's module setting, especially this part:

You very likely want "CommonJS" for node projects.

I'm going to close this issue because it's not a problem with esbuild. The build script for that package is misconfigured.

jamescrowley commented 3 years ago

Thanks @evanw much appreciated