Open deedubFICTIV opened 11 months ago
deedubFICTIV
commented
11 months ago Updating to v3.3.0 got me past the issues I was having.
I'll leave this open in case someone wants to look into why those issues were experienced with v4.20.0, but for those stuck where I was, v3.3.0 will likely be what you need
rattrayalex
commented
11 months ago Can you share tsconfig, package.json, and jest config?
If you are using @jest-environment jsdom, there's a good chance that you actually want to use import 'openai/shims/node' and add a global fetch polyfill, for example with undici, instead. But it depends on your setup.
Note that v3.3.0 has many downsides, like no auto-retry, outdated/incorrect types, no support for streaming, etc.
deedubFICTIV
commented
11 months ago I did use openai/shims/node which got me past the fetch error.
Here's the tsconfig.json file:
// tsconfig.json
{
"compilerOptions": {
/* Basic Options */
//"incremental": true, /* Enable incremental compilation */
"target": "es2019" /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019' or 'ESNEXT'. */,
"module": "commonjs" /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */,
// "lib": [], /* Specify library files to be included in the compilation. */
"allowJs": true /* Allow javascript files to be compiled. */,
"checkJs": false /* Report errors in .js files. */,
"jsx": "react" /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */,
"declaration": true /* Generates corresponding '.d.ts' file. */,
"declarationMap": true /* Generates a sourcemap for each corresponding '.d.ts' file. */,
"sourceMap": true /* Generates corresponding '.map' file. */,
// "outFile": "./", /* Concatenate and emit output to single file. */
// "outDir": "./dist", /* Redirect output structure to the directory. */
"rootDir": "." /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */,
"composite": true /* Enable project compilation */,
// "tsBuildInfoFile": "./", /* Specify file to store incremental compilation information */
// "removeComments": true, /* Do not emit comments to output. */
// "noEmit": true, /* Do not emit outputs. */
// "importHelpers": true, /* Import emit helpers from 'tslib'. */
// "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */
// "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */
/* Strict Type-Checking Options */
// "strict": true, /* Enable all strict type-checking options. */
// "noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
"strictNullChecks": true /* Enable strict null checks. */,
"strictFunctionTypes": true /* Enable strict checking of function types. */,
"strictBindCallApply": true /* Enable strict 'bind', 'call', and 'apply' methods on functions. */,
"strictPropertyInitialization": true /* Enable strict checking of property initialization in classes. */,
"noImplicitThis": true /* Raise error on 'this' expressions with an implied 'any' type. */,
"alwaysStrict": true /* Parse in strict mode and emit "use strict" for each source file. */,
/* Additional Checks */
// "noUnusedLocals": true, /* Report errors on unused locals. */
// "noUnusedParameters": true, /* Report errors on unused parameters. */
// "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */
// "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */
/* Module Resolution Options */
"moduleResolution": "node" /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */,
"baseUrl": "./" /* Base directory to resolve non-absolute module names. */,
"paths": {
/* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */
"@fictiv/*": ["./packages/*"]
},
// "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */
"typeRoots": [
"node_modules/@types"
] /* List of folders to include type definitions from. */,
// "types": [], /* Type declaration files to be included in compilation. */
"allowSyntheticDefaultImports": true /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */,
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */,
// "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */
// "allowUmdGlobalAccess": true, /* Allow accessing UMD globals from modules. */
/* Source Map Options */
// "sourceRoot": "", /* Specify the location where debugger should locate TypeScript files instead of source locations. */
// "mapRoot": "", /* Specify the location where debugger should locate map files instead of generated locations. */
// "inlineSourceMap": true /* Emit a single file with source maps instead of having a separate file. */,
// "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */
/* Experimental Options */
// "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */
// "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */
/* Advanced Options */
"forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */,
"resolveJsonModule": true,
"skipLibCheck": true
},
"exclude": [
"**/node_modules/**/*",
"**/dist/**/*",
"./_templates/**/*",
"./bin/**",
"./coverage/**",
"./eslint-custom-rules/**",
"./babel.config.js"
]
}And here's package.json which includes the jest config (I removed scripts and other private info)
// package.json
{
"version": "1.0.0",
"license": "UNLICENSED",
"private": true,
"workspaces": [
"packages/*",
"services/*"
],
"jest": {
"preset": "ts-jest",
"collectCoverageFrom": [
"services/**/*.{js,ts,tsx}",
"packages/**/*.{js,ts,tsx}"
],
"transform": {
"^.+\\.(ts|tsx)$": [
"ts-jest",
{
"isolatedModules": true
}
]
},
"coveragePathIgnorePatterns": [
"/node_modules/",
"/db/",
"/dist/",
"__mocks__",
"__tests__",
"test-helpers/",
"/.build/",
"packages/static-data/"
],
"moduleNameMapper": {
"~/graphql-server/(.*)$": "<rootDir>/services/graphql-server/src/$1",
"~/zip-file-service/(.*)$": "<rootDir>/services/zip-file-service/src/$1",
"~/zapier-events/(.*)$": "<rootDir>/services/zapier-events/src/$1",
"~/kinesis-monitoring/(.*)$": "<rootDir>/services/kinesis-monitoring/src/$1",
"^uuid$": "uuid"
},
"transformIgnorePatterns": [
"/node_modules/(?!jest-when|formdata-node)"
],
"testPathIgnorePatterns": [
"<rootDir>/dist/",
"/dist/",
"/node_modules/",
"/.build/",
"<rootDir>/*/.*/__tests__/test-helpers"
],
"moduleDirectories": [
"node_modules",
"src"
],
"moduleFileExtensions": [
"ts",
"tsx",
"js",
"jsx",
"json"
],
"modulePathIgnorePatterns": [
"<rootDir>/services/src/*/.*/__mocks__",
"/dist/"
],
"testEnvironment": "node",
"setupFilesAfterEnv": [
"<rootDir>/packages/lib-test-helpers/src/unitTestSetup.js"
],
"reporters": [
"default",
"jest-junit"
],
"coverageReporters": [
"text",
"html"
],
"snapshotSerializers": [
"<rootDir>/node_modules/@whoaa/jest-mock-knex/serializer"
],
"roots": [
"<rootDir>",
"<rootDir>/packages/",
"<rootDir>/services/"
],
"testTimeout": 300000
},
"jest-junit": {
"outputDirectory": "./reports",
"outputName": "jest.xml"
},
"devDependencies": {
"@aws-sdk/client-s3": "~3.379.1",
"@pdftron/pdfnet-node": "^9.2.0",
"@types/aws-lambda": "^8.10.85",
"@types/bunyan": "^1.8.6",
"@types/express": "^4.17.2",
"@types/express-serve-static-core": "^4.17.1",
"@types/jest": "~27.4.1",
"@types/jsforce": "^1.11.0",
"@types/jsonwebtoken": "^7.2.7",
"@types/lodash": "^4.14.133",
"@types/node": "~13.7.0",
"@types/puppeteer": "~5.4.3",
"@typescript-eslint/eslint-plugin": "^5.61.0",
"@typescript-eslint/parser": "^5.61.0",
"@whoaa/jest-mock-knex": "^1.20.0",
"add": "^2.0.6",
"apollo-cache-inmemory": "^1.1.12",
"apollo-client": "^2.2.8",
"apollo-link": "^1.2.1",
"apollo-link-http": "^1.5.3",
"aws-sdk": "~2.1374.0",
"brfs": "^2.0.2",
"chalk": "^2.4.1",
"concurrently": "^6.5.1",
"copy-webpack-plugin": "^9.0.0",
"db-migrate": "^0.11.13",
"db-migrate-pg": "^1.2.2",
"delay": "^4.3.0",
"dogapi": "^2.8.4",
"eslint": "~8.47.0",
"eslint-config-prettier": "~8.5.0",
"eslint-import-resolver-custom-alias": "~1.3.2",
"eslint-plugin-fictiv-apis": "file:./eslint-custom-rules",
"eslint-plugin-import": "~2.28.1",
"eslint-plugin-jest": "~27.2.3",
"eslint-plugin-no-only-tests": "~3.1.0",
"eslint-plugin-node": "~11.1.0",
"eslint-plugin-prettier": "~5.0.0",
"eslint-plugin-react": "~7.33.2",
"eslint-plugin-simple-import-sort": "~10.0.0",
"expect-more-jest": "~5.5.0",
"graphql": "^15.5.1",
"graphql-fields": "^1.0.2",
"graphql-tools": "^4.0.7",
"jest": "~29.7.0",
"jest-junit": "~16.0.0",
"jest-when": "~3.5.1",
"js-yaml": "~4.1.0",
"json-loader": "^0.5.4",
"lerna": "~6.6.2",
"mock-knex": "~0.4.10",
"mockdate": "^3.0.2",
"node-loader": "^2.0.0",
"node-loader-relative": "^0.0.3",
"nodemon": "^3.0.1",
"nope-sorry": "^1.0.0",
"npm-conf": "^1.1.3",
"prettier": "^3.0.2",
"quick-lru": "^4.0.1",
"raw-loader": "^0.5.1",
"rimraf": "^3.0.1",
"serverless": "~3.34.0",
"serverless-http": "~3.2.0",
"serverless-offline": "~12.0.4",
"serverless-package-python-functions": "^0.7.0",
"serverless-plugin-git-variables": "^5.2.0",
"serverless-webpack": "~5.13.0",
"stripe": "^12.6.0",
"supertest": "^6.3.3",
"transform-loader": "^0.2.4",
"ts-jest": "^29.1.1",
"ts-loader": "^9.2.6",
"ts-node": "^10.9.1",
"typescript": "~5.1.6",
"utility-types": "^3.10.0",
"webpack": "~5.82.0",
"webpack-node-externals": "~3.0.0",
"yaml-jest": "~1.2.0",
"yargs": "^17.3.1",
"yarn": "^1.21.1",
"yml-loader": "^2.1.0"
},
"resolutions": {
"pg": "^8.11.1",
"brfs": "^2.0.2"
},
"dependencies": {
"@opensearch-project/opensearch": "^2.0.0",
"@easypost/api": "^5.10.1",
"ajv": "^6.12.3",
"analytics-node": "6.2.0",
"apollo-server-express": "^3.12.0",
"aws-sdk": "~2.1374.0",
"aws-opensearch-connector": "^1.1.0",
"axios": "~1.6.2",
"base64-url": "^2.0.0",
"bluebird": "^3.5.0",
"bunyan": "^1.8.10",
"bunyan-middleware": "^1.0.1",
"cli": "^1.0.1",
"cognito-express": "~3.0.2",
"cookie-parser": "1.4.5",
"cors": "^2.8.3",
"dataloader": "^1.4.0",
"decimal.js": "^7.2.0",
"dotenv": "^4.0.0",
"dynamodb-data-types": "^3.0.0",
"dynamodb-migrations": "^0.0.10",
"express": "^4.18.2",
"express-jwt-authz": "^1.0.0",
"graphql": "^15.5.1",
"graphql-tools": "^4.0.7",
"graphql-apollo-errors": "^2.0.3",
"graphql-custom-types": "^1.5.1",
"graphql-extensions": "^0.10.10",
"graphql-resolve-batch": "^1.0.2",
"graphql-type-json": "^0.3.1",
"helmet": "^4.1.1",
"i18n-iso-countries": "6.0.0",
"is-uuid": "^1.0.2",
"joi": "17.2.1",
"jsforce": "^1.11.1",
"jsonwebtoken": "9.0.0",
"jsx-pdf": "whoaa512/jsx-pdf#58feebf",
"knex": "^0.21.21",
"libphonenumber-js": "^1.9.44",
"lodash": "^4.17.21",
"moment": "~2.29.4",
"moment-timezone": "~0.5.37",
"nocache": "^2.1.0",
"odoo-connect": "https://github.com/fictiv/odoo-connect.git#v1.3.0",
"openai": "3.3.0",
"p-iteration": "^1.1.8",
"p-map": "^2.1.0",
"p-props": "^2.0.0",
"p-reduce": "^1.0.0",
"pdfmake": "^0.2.7",
"pdf-merger-js": "^4.3.0",
"pg": "^8.11.1",
"postal-codes-js": "2.4.0",
"psl": "1.8.0",
"raven": "^2.6.0",
"@sentry/node": "5.15.4",
"source-map-support": "^0.4.15",
"stripe": "^8.154.0",
"url-join": "^4.0.0",
"uuid": "^2.0.1",
"jwks-rsa": "^2.0.5",
"json2csv": "6.0.0-alpha.2",
"qrcode": "^1.5.1",
"@sideway/address": "4.1.2",
"xml-js": "1.6.11",
"xml2js": "~0.6.2",
"vwo-node-sdk": "^1.62.0",
"whatwg-fetch": "3.6.19",
"tough-cookie": "^4.1.3",
"request": "^2.88.2"
}
}Ah I'm so sorry, I meant openai/shims/web not openai/shims/node in that case.
deedubFICTIV
commented
11 months ago To be clear, this was being used in a backend nodeJS environment. Would I still want to use the web shim?
rattrayalex
commented
11 months ago Yes, you can as long as you have access to the global fetch function – it'll just use that. However, file uploads will require you to also polyfill FormData, File, and other classes – I think undici exposes these.
RobertHaslinger
commented
10 months ago I first encountered the fetch is not defined bug, which led me to #304 and was able to move past the issue with
import 'openai/shims/node'
I also encountered the same problem with:
node: 20.10.0openai: 4.24.1jest: 29.4.2testEnvironment: 'node'I managed to get the tests working by first removing the line import 'openai/shims/node' again.
Then, to fix the "ReferenceError: fetch is not defined" error, I did the following:
undici as devDependency (6.2.1 in my case)jest.setup.js file (see https://github.com/mswjs/msw/discussions/1934#discussioncomment-7874204):
//jest.setup.js
const { TextDecoder, TextEncoder, ReadableStream } = require("node:util")Object.defineProperties(globalThis, { TextDecoder: { value: TextDecoder }, TextEncoder: { value: TextEncoder }, ReadableStream: { value: ReadableStream }, })
const { Blob, File } = require("node:buffer") const { fetch, Headers, FormData, Request, Response } = require("undici")
Object.defineProperties(globalThis, { fetch: { value: fetch, writable: true }, Blob: { value: Blob }, File: { value: File }, Headers: { value: Headers }, FormData: { value: FormData }, Request: { value: Request }, Response: { value: Response }, })
3. Reference this setup in the `jest.config.js`:
```js
//jest.config.js
module.exports = {
setupFilesAfterEnv: ['./jest.setup.js'],
...other config
}
Fiattarone
commented
10 months ago Right on @RobertHaslinger, nailed it. I just needed Request and Response in my case, but great fix.
Confirm this is a Node library issue and not an underlying OpenAI API issue
Describe the bug
Issues experienced while running unit tests in our nodejs backend. I first encountered the fetch is not defined bug, which led me to https://github.com/openai/openai-node/issues/304 and was able to move past the issue with
import 'openai/shims/node'But now I'm receiving the following errors:
I've also added
But I'll still get the same
SyntaxError: Unexpected token 'export'error.I've updated the
transformIgnorePatternsvalue to include"/node_modules/(?!formdata-node)"as well as"/node_modules/(?!openai/node/shims)"and many combinations. Still no luck.yarn v1.22.4 typescript v5.1.6 jest v29.7.0
To Reproduce
yarn add openaiI've been able to reproduce it with as little as just the imports. Add the following imports to a file that has unit tests (or a logic/utils file used by a file being tested)
run the tests on that file
Code snippets
No response
OS
macOS
Node version
Node v16.20.0 (also tested with v18.19.0 same results)
Library version
openai v4.20.0