ECONNREFUSED on Mac M1 when following time travel debugger quickstart

[timr11]

When following the docs for the time traveller debug quickstart, it gave me a schema validation error for the dbos-config.yaml when hitting the "Time Travel Debug" link due to the PGPASSWORD. I set it manually and got a ECONNREFUSED when connecting to the debugger proxy. Spoke with @devhawk about it, repro steps below:

> npx -y @dbos-inc/create@latest -n dbos-time-travel-debugger
> cd dbos-time-travel-debugger/
> bun ./start_postgres_docker.js
Waiting for PostgreSQL to start...
PostgreSQL started!
Database started successfully!

> npx dbos migrate
2024-06-05 00:19:37 [info]: Starting migration: creating database hello if it does not exist 
2024-06-05 00:19:37 [info]: Database hello does not exist, creating... 
2024-06-05 00:19:37 [info]: Executing migration command: npx knex migrate:latest 
2024-06-05 00:19:37 [info]: Batch 1 run: 1 migrations 
2024-06-05 00:19:37 [info]: Creating DBOS tables and system database. 
2024-06-05 00:19:38 [info]: Migration successful!

Then I copied over the operations.ts from the tutorial and ran:

> npm run build
> npx dbos-cloud login
2024-06-05 00:10:51 [info]: Please authenticate with DBOS Cloud! 
Login URL: https://login.dbos.dev/activate?user_code=TXPX-SLQZ
2024-06-05 00:10:57 [info]: Waiting for login... 
2024-06-05 00:11:08 [info]: Successfully logged in as timromanski!

> npx dbos-cloud db provision ttd-db -U timr11
* Password: abcdefgh
2024-06-05 00:12:05 [info]: Successfully started provisioning database: ttd-db 
2024-06-05 00:12:11 [info]: {"PostgresInstanceName":"ttd-db","HostName":"userdb-ca3e9ffc-a922-47cd-94f1-cf75a876a1d8.cvc4gmaa6qm9.us-east-1.rds.amazonaws.com","Status":"available","Port":5432,"DatabaseUsername":"timr11","AdminUsername":"timr11"} 
2024-06-05 00:12:11 [info]: Database successfully provisioned!

> npx dbos-cloud app register -d ttd-db
2024-06-05 00:13:56 [info]: Loaded application name from package.json: dbos-time-travel-debugger 
2024-06-05 00:13:56 [info]: Registering application: dbos-time-travel-debugger 
2024-06-05 00:13:57 [info]: dbos-time-travel-debugger ID: 5b5287a8-cc11-4e40-9b75-1542432f27eb 
2024-06-05 00:13:57 [info]: Successfully registered dbos-time-travel-debugger!

> npx dbos-cloud app deploy
2024-06-04 18:03:55 [info]: Loaded application name from package.json: dbos-time-travel-debugger 
2024-06-04 18:03:55 [info]: Submitting deploy request for dbos-time-travel-debugger 
2024-06-04 18:04:30 [info]: Submitted deploy request for dbos-time-travel-debugger. Assigned version: 1717524235 
2024-06-04 18:04:39 [info]: Successfully deployed dbos-time-travel-debugger! 
2024-06-04 18:04:39 [info]: Access your application at https://timromanski-dbos-time-travel-debugger.cloud.dbos.dev/

Then I navigated to https://timromanski-dbos-time-travel-debugger.cloud.dbos.dev/greeting/tim and saw the application was working. I hit the “Time Travel Debug” link above the GreetingWorkflow, entered the DB password when prompted and selected the workflow ID associated with my website visit from earlier, it ran the following in my terminal:

> npx dbos-sdk debug -x http://localhost:2345 -u 6cd2dbbc-a258-4572-a8e1-11d29e0c4296
Debugger attached.
Debugger attached.
Waiting for the debugger to disconnect...
/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/dbos-runtime/config.js:143
        throw new error_1.DBOSInitializationError(`dbos-config.yaml failed schema validation. ${errorMessages}`);
              ^

DBOSInitializationError: dbos-config.yaml failed schema validation. Error: must be string (schema path: #/properties/database/properties/password/type)
    at parseConfigFile (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/dbos-runtime/config.js:143:15)
    at Command.<anonymous> (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/dbos-runtime/cli.js:42:70)
    at Command.listener [as _actionHandler] (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:523:17)
    at /Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1375:65
    at Command._chainOrCall (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1272:12)
    at Command._parseCommand (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1375:27)
    at /Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1161:27
    at Command._chainOrCall (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1272:12)
    at Command._dispatchSubcommand (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1157:25)
    at Command._parseCommand (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/commander/lib/command.js:1343:19) {
  dbosErrorCode: 3
}

Node.js v18.18.2
Waiting for the debugger to disconnect...

I set the PGPASSWORD env var in the terminal and reran the command, got:

> npx dbos-sdk debug -x http://localhost:2345 -u 6cd2dbbc-a258-4572-a8e1-11d29e0c4296
Debugger attached.
Debugger attached.
2024-06-05 01:31:36 [info]: Running in debug mode! 
2024-06-05 01:31:36 [info]: Debugging mode proxy: localhost:2345 
2024-06-05 01:31:36 [info]: Workflow executor initialized 
2024-06-05 01:31:36 [error]: Debug mode error: connect ECONNREFUSED ::1:2345 
    at GlobalLogger.error (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/telemetry/logs.js:59:65)
    at debugWorkflow (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/dbos-runtime/debug.js:24:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Command.<anonymous> (/Users/timromanski/github/dbos-time-travel-debugger/node_modules/@dbos-inc/dbos-sdk/dist/src/dbos-runtime/cli.js:43:5)
Waiting for the debugger to disconnect...
Waiting for the debugger to disconnect...

I checked and saw the proxy server was running:

> lsof -i -P -n | grep 2345
debug-pro 18455 timromanski    9u  IPv4 0xfbffa68d9ddfcb65      0t0  TCP 127.0.0.1:2345 (LISTEN)

Would love to get the time traveller debugger working, it's very cool. Help appreciated!

Environment:

• Mac M1, macOS v13.6.7 (22G720)
• IDE is cursor, a VS Code fork

[apoliakov]

Hi @timr11! Sorry about the trouble.

It looks like there's possibly several issues.

1. Known issue: schema checker aggressively rejects $PGPASSWORD.

This was actually fixed two commits ago: https://github.com/dbos-inc/dbos-transact/commit/208c001e4c0f5423594e352c09d63a8ad7321f56 The trick is, the password: that is set in dbos-config.yaml is only used for when you run the app locally, not in debug mode. Otherwise it's only typechecked aggressively but otherwise doesn't matter. So the simple workaround for this issue is to set it to virtually any string, like password:''.

2. Your ECONNREFUSED

This one I currently don't have an explanation for. Let's see if we can gather more data. I can give you some things to try:

1. With the proxy running, get psql on your system and try to run psql -h localhost -p 2345 Does that work? "Work" here means it looks like a postgres connection that lets you run queries. This connection should not need a password. If it gives you an error message, let us know what it is.

2. Try resetting VSCode credentials. In your IDE, hit Command + Shift + P. A command palette should pop up. Start typing in "DBOS". You should see

   • DBOS: Delete Stored Application Database Password
   • DBOS: Delete Stored DBOS Cloud Credentials Try running these in case you mistyped your password. Then try debug again.

3. Try Debug from the dashboard. Run npx dbos-cloud dashboard launch Go to the URL of the dashboard. Find a workflow, click on it's workflow uuid and select "Debug this workflow". Normally that takes you to VSCode and starts a debugger. Maybe this way will work better for you. See also: https://docs.dbos.dev/cloud-tutorials/monitoring-dashboard

See if these help? Meanwhile I'll see if anyone else has encountered this ECONNREFUSED error before.

[timr11]

Hi @apoliakov, thank you for your response! On the ECONNREFUSED, I followed your suggestion and was able to connect via psql -h localhost -p 2345. I'm able to list the tables and run queries on them. I also reset the credentials as you suggested, and still no dice. I tried debugging from the dashboard, I found a workflow, clicked on its ID and selected "Debug this workflow", and it gave me an error in VS Code that the Workflow ID ... not found. This happens both on Cursor and VS Code.

[apoliakov]

Thanks @timr11 !

The psql connection should allow you interactive time travel - item 1 in the recent blog post: https://www.dbos.dev/blog/database-time-travel . It's a neat capability to try out because it lets you examine past database state using a specific time stamp.

If that works, it means the proxy connection is healthy. But, for some reason, VSCode has trouble finding your workflows. And, to confirm, you have the app's directory open in VSCode, right?

@devhawk mentioned he may have some advanced maneuvers we can try out here. I'll let him opine.

[qianl15]

It looks like the debug mode was trying to connect the proxy through IPV6 but the proxy itself serves with IPV4.

The address here ::1 is IPV6

Debug mode error: connect ECONNREFUSED ::1:2345 

The proxy listens to 127.0.0.1 which is IPV4

> lsof -i -P -n | grep 2345 debug-pro 18455 timromanski 9u IPv4 0xfbffa68d9ddfcb65 0t0 TCP 127.0.0.1:2345 (LISTEN)

It might have something to do with the underlying config on your laptop. If you try to run npx dbos-sdk debug -x http://127.0.0.1:2345 -u 6cd2dbbc-a258-4572-a8e1-11d29e0c4296 it should succeed.

Meanwhile, we're fixing on our end to make sure the proxy listens to both IPv4 and IPv6. Thanks for reporting this issue!

[qianl15]

Hi @apoliakov, thank you for your response! On the ECONNREFUSED, I followed your suggestion and was able to connect via psql -h localhost -p 2345. I'm able to list the tables and run queries on them. I also reset the credentials as you suggested, and still no dice. I tried debugging from the dashboard, I found a workflow, clicked on its ID and selected "Debug this workflow", and it gave me an error in VS Code that the Workflow ID ... not found. This happens both on Cursor and VS Code.

Regarding this issue, did you delete your DB instance and recreate one? Because time travel information would be cleared when you destroy a DB instance.

[timr11]

Update, I tried again today and it worked! @qianl15 when selecting "Time Travel Debug" it ran npx dbos debug -x postgresql://localhost:2345 -u 7b71c416-a135-47a2-8e6f-cf8c3aae452e, I see that your fix changed "http://localhost.." to "postgresql://localhost.." so I'm assuming that that did the trick. Thank you for the help, greatly appreciate it! This is really cool