search

typegoose / mongodb-memory-server

Manage & spin up mongodb server binaries with zero(or slight) configuration for tests.
https://typegoose.github.io/mongodb-memory-server/
MIT License
2.58k stars 187 forks source link

Problem running with Github Actions using Bun #791

Closed clburlison closed 1 year ago

clburlison commented 1 year ago

Versions

package: mongo-memory-server

What is the Problem?

When running MMS via Github Actions I'm receiving a TypeError on _StartUpInstance (full debug log below). This same environment works on my local developer system that is running macOS arm64. I know bun.sh is still very much an alpha product but the fact that this exact same code is working on my local mac makes me think there might be some kind of configuration issue.

Code Example

.github/workflows/checks.yml

name: Checks

on:
  workflow_dispatch:
  pull_request:
    branches:
      - main

env:
  MONGOMS_DEBUG: '1'
  MONGOMS_VERSION: 6.0.9

jobs:
  checks:
    strategy:
      fail-fast: false
      matrix:
        task:
          - name: Run tests
            command: 'cd server && bun test'

    runs-on: ubuntu-latest

    steps:
      - name: Check out Git repository
        uses: actions/checkout@v3

      - uses: oven-sh/setup-bun@1c8c9a76158ac02fec2d670c3deb0dca37650714 # v1
        with:
          bun-version: 0.8.1

      - name: Install bun dependencies
        run: |
          bun install --cwd server

      - name: Run job
        run: ${{ matrix.task.command }}
        env:
          MONGOMS_DEBUG: ${{ env.MONGOMS_DEBUG }}
          MONGOMS_VERSION: ${{ env.MONGOMS_VERSION }}

server/tests/example.test.ts

import { MongoMemoryServer } from 'mongodb-memory-server'

const instance = await MongoMemoryServer.create({
  instance: {
    dbName: 'test',
    storageEngine: 'wiredTiger',
  },
})
const uri = instance.getUri()
console.log(uri)
await instance.stop()

Debug Output

Debug Output ```txt cd server && bun test shell: /usr/bin/bash -e {0} env: MONGOMS_DEBUG: 1 MONGOMS_VERSION: 6.0.9 bun test v0.8.1 (16b4bf34) tests/routes/scim.test.ts: 2023-08-30T15:04:49.844Z MongoMS:ResolveConfig Debug Mode Enabled, through Environment Variable 2023-08-30T15:04:49.846Z MongoMS:ResolveConfig findPackageJson: Found package.json at "/home/runner/work/project-name/project-name/server/package.json" 2023-08-30T15:04:49.847Z MongoMS:ResolveConfig findPackageJson: Found package.json at "/home/runner/work/project-name/project-name/package.json" 2023-08-30T15:04:50.609Z MongoMS:MongoMemoryServer create: Called .create() method 2023-08-30T15:04:50.610Z MongoMS:MongoMemoryServer Mongo[unknown]: start: Called .start() method 2023-08-30T15:04:50.611Z MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Called MongoMemoryServer._startUpInstance() method 2023-08-30T15:04:50.611Z MongoMS:MongoMemoryServer Mongo[unknown]: getStartOptions: forceSamePort: false 2023-08-30T15:04:50.619Z MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Creating new MongoDB instance with options: { instance: { port: 33579, dbName: 'test', ip: '127.0.0.1', storageEngine: 'wiredTiger', replSet: undefined, dbPath: '/tmp/mongo-mem-qbLxKJ', tmpDir: '/tmp/mongo-mem-qbLxKJ', keyfileLocation: undefined, launchTimeout: undefined, args: undefined, auth: false }, binary: undefined, spawn: undefined } 2023-08-30T15:04:50.621Z MongoMS:MongoInstance create: Called .create() method 2023-08-30T15:04:50.621Z MongoMS:MongoInstance Mongo[33579]: start 2023-08-30T15:04:50.622Z MongoMS:MongoBinary getPath 2023-08-30T15:04:50.622Z MongoMS:DryMongoBinary generateOptions 2023-08-30T15:04:50.623Z MongoMS:utils tryReleaseFile: "/etc/upstream-release/lsb-release" does not exist 2023-08-30T15:04:50.624Z MongoMS:getos getLinuxInformation: Using etcOsRelease 2023-08-30T15:04:50.624Z MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" 2023-08-30T15:04:50.625Z MongoMS:DryMongoBinary generatePaths { version: '6.0.9', downloadDir: '.cache/mongodb-binaries', os: { os: 'linux', dist: 'ubuntu', codename: 'jammy', release: '22.04', id_like: [ 'debian' ] }, platform: 'linux', arch: 'x64', 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "6.0.9" was found 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary generateDownloadPath: using resolveConfig (DOWNLOAD_DIR) "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9" 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary generateOptions: ARCHIVE_NAME or DOWNLOAD_URL defined, generating options based on that (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz") 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary parseArchiveNameRegex (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz") 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary getBinaryName 2023-08-30T15:04:50.650Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:50.653Z MongoMS:MongoBinaryDownload startDownload 2023-08-30T15:04:50.653Z MongoMS:MongoBinaryDownloadUrl Using "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz" as the Download-URL 2023-08-30T15:04:50.654Z MongoMS:MongoBinaryDownload download 2023-08-30T15:04:50.655Z MongoMS:MongoBinaryDownload download: Downloading: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz" 2023-08-30T15:04:50.655Z MongoMS:MongoBinaryDownload httpDownload 2023-08-30T15:04:50.656Z MongoMS:MongoBinaryDownload httpDownload: trying to download "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz" Downloading MongoDB "6.0.9": 0.1% (0mb / 69.2mb) Downloading MongoDB "6.0.9": 100% (69.2mb / 69.2mb) 2023-08-30T15:04:50.977Z MongoMS:MongoBinaryDownload httpDownload: moved "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz.downloading" to "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz" 2023-08-30T15:04:50.978Z MongoMS:MongoBinaryDownload makeMD5check: Checking MD5 of downloaded binary... 2023-08-30T15:04:50.978Z MongoMS:MongoBinaryDownload makeMD5check: checkMD5 is disabled 2023-08-30T15:04:50.979Z MongoMS:MongoBinaryDownload extract 2023-08-30T15:04:50.979Z MongoMS:DryMongoBinary generateOptions 2023-08-30T15:04:50.979Z MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" 2023-08-30T15:04:50.979Z MongoMS:DryMongoBinary generatePaths { version: '6.0.9', downloadDir: '.cache/mongodb-binaries', os: { os: 'linux', dist: 'ubuntu', release: '' }, platform: 'linux', arch: 'x86_64', systemBinary: '' } 2023-08-30T15:04:50.980Z MongoMS:DryMongoBinary getBinaryName 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:50.981Z MongoMS:DryMongoBinary generateDownloadPath: Paths: { legacyHomeCache: '/home/runner/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9', modulesCache: '/home/runner/work/project-name/project-name/server/node_modules/.cache/mongodb-memory-server/mongod-x86_64-ubuntu-6.0.9', relative: '/home/runner/work/project-name/project-name/server/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9', resolveConfig: '/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9' } 2023-08-30T15:04:50.996Z MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "6.0.9" was found 2023-08-30T15:04:50.996Z MongoMS:DryMongoBinary generateDownloadPath: using resolveConfig (DOWNLOAD_DIR) "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9" 2023-08-30T15:04:50.996Z MongoMS:DryMongoBinary generateOptions: ARCHIVE_NAME or DOWNLOAD_URL defined, generating options based on that (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz") 2023-08-30T15:04:50.997Z MongoMS:DryMongoBinary parseArchiveNameRegex (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz") 2023-08-30T15:04:50.998Z MongoMS:DryMongoBinary getBinaryName 2023-08-30T15:04:50.998Z MongoMS:DryMongoBinary combineBinaryName 2023-08-30T15:04:51.002Z MongoMS:MongoBinaryDownload extract: archive: "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgz" final: "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9" 2023-08-30T15:04:51.002Z MongoMS:MongoBinaryDownload extractTarGz 2023-08-30T15:04:54.110Z MongoMS:MongoBinary download: Removing Download lock 2023-08-30T15:04:54.111Z MongoMS:LockFile unlock: Unlocking file "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/6.0.9.lock" 2023-08-30T15:04:54.111Z MongoMS:LockFile checkLock: for file "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/6.0.9.lock" with uuid: "5c6c9c0a-7702-4299-be02-496ba29815a8" 2023-08-30T15:04:54.111Z MongoMS:LockFile checkLock: Lock File Already exists, and is for *this* process, with uuid: "5c6c9c0a-7702-4299-be02-496ba29815a8" 2023-08-30T15:04:54.111Z MongoMS:LockFile unlock: Lock Status was "availableInstance" for file "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/6.0.9.lock" 2023-08-30T15:04:54.112Z MongoMS:LockFile unlockCleanup: for file "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/6.0.9.lock" 2023-08-30T15:04:54.112Z MongoMS:MongoBinary download: Download lock removed 2023-08-30T15:04:54.112Z MongoMS:MongoBinary getPath: Mongod binary path: "/home/runner/work/project-name/project-name/server/.cache/mongodb-binaries/mongod-x86_64-ubuntu-6.0.9" 2023-08-30T15:04:54.112Z MongoMS:MongoInstance Mongo[33579]: start: Starting Processes 2023-08-30T15:04:54.113Z MongoMS:MongoInstance Mongo[33579]: _launchMongod: Launching Mongod Process 2023-08-30T15:04:54.113Z MongoMS:MongoInstance Mongo[33579]: prepareCommandArgs 2023-08-30T15:04:54.113Z MongoMS:MongoInstance Mongo[33579]: prepareCommandArgs: final argument array:["--port","33579","--dbpath","/tmp/mongo-mem-qbLxKJ","--storageEngine","wiredTiger","--bind_ip","127.0.0.1","--noauth"] 2023-08-30T15:04:54.115Z MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance threw a Error: [TypeError: The "readableStream" argument must be an instance of ReadableStream. Received undefined] 2023-08-30T15:04:54.115Z MongoMS:MongoMemoryServer Mongo[unknown]: stop: Called .stop() method 2023-08-30T15:04:54.116Z MongoMS:MongoMemoryServer Mongo[unknown]: stop: "instanceInfo" is not defined (never ran?) TypeError: The "readableStream" argument must be an instance of ReadableStream. Received undefined code: "ERR_INVALID_ARG_TYPE" at node:stream:373:20 ```

Do you know why it happens?

no

I was also able to replicate this on a local Ubuntu 22.04 linux VM. The same run command fails with the same readableStream error message. However if I look at the cache directory the binary does exist. If I run the command a second it works as intended.

MONGOMS_DEBUG=1 MONGOMS_DOWNLOAD_DIR=/root/bun-testing/.cache MONGOMS_VERSION=6.0.9 bun run example.test.ts
clburlison commented 1 year ago

Closing as I've confirmed this is an error with bun. Installing node v18 and using that runtime worked. It's still strange that is only occurs on Ubuntu mean while my macOS host works every time on the first time.

MONGOMS_DEBUG=1 MONGOMS_DOWNLOAD_DIR=/root/bun-testing/.cache MONGOMS_VERSION=6.0.8 node example.js

const MongoMemoryServer = require('mongodb-memory-server').MongoMemoryServer 

async function main() {
  const instance = await MongoMemoryServer.create({
    instance: {
      dbName: 'test',
      storageEngine: 'wiredTiger',
    },
  })
  const uri = instance.getUri()
  console.log(uri)
  await instance.stop()
}

main()
  .then(() => console.log('done'))
  .catch(console.error)

Thanks for the great project and sorry for the noise. Thought I had ran through all the possible iterations of tests prior to opening.

hasezoey commented 1 year ago

it is definitely a problem with bun, to be more specific, it seems like bun fails to start a child-process, because it fails on the following line (or very close tho that line):

https://github.com/nodkz/mongodb-memory-server/blob/f529e11c95431c8e66b95404a567c468ba689e75/packages/mongodb-memory-server-core/src/util/MongoInstance.ts#L474-L477

robertherber commented 1 year ago

Tried to find an issue in Bun tracking this - have you seen one @clburlison @hasezoey?

hasezoey commented 1 year ago

Tried to find an issue in Bun tracking this - have you seen one

i have not seen one, i also dont use bun and dont quite know what the issue is actually about, so i cannot make a proper bug report about it

robertherber commented 1 year ago

My current workaround is simply to run MacOS on Github Actions.. 😅

robertherber commented 11 months ago

This is an issue when running on Linux in any environment as far as I can tell. @clburlison could you provide some more info on what kind of Bun issue it is? I'd like to make sure it's tracked somewhere - but don't really know where to start since I can't reproduce it locally! :)