Closed vincekruger closed 8 months ago
Just wanted to mention that require('dotenv').config() is not needed in bun
๐๐ผ It's such an old habit ๐
I get the same error with google cloud pubsub. I was debugging the functio to at least get the error which is a little bit more helpful:
error at makeUnaryRequest
Not sure yours is the same but it might help. You can use bun --inspect
and put a breakpoint in the function getErrorStackString
Seems like a bug with how bun handles error.stack ?
For now I could get over this by adding a patch with patch-package for @grpc/grpc-js
Steps:
bun add --dev patch-package
"scripts": {
...
...
"postinstall": "patch-package"
},
patches/@grpc+grpc-js+1.8.21.patch
diff --git a/node_modules/@grpc/grpc-js/build/src/client.js b/node_modules/@grpc/grpc-js/build/src/client.js
index 79b1adc..5302492 100644
--- a/node_modules/@grpc/grpc-js/build/src/client.js
+++ b/node_modules/@grpc/grpc-js/build/src/client.js
@@ -31,7 +31,7 @@ function isFunction(arg) {
return typeof arg === 'function';
}
function getErrorStackString(error) {
- return error.stack.split('\n').slice(1).join('\n');
+ return error?.stack?.split('\n')?.slice(1)?.join('\n');
}
/**
* A generic gRPC client. Primarily useful as a base class for all generated
4. bun install
It should print
bun install v1.0.0 (822a00c4) patch-package 8.0.0 Applying patches... @grpc/grpc-js@1.8.21 โ
5. Run your project
I'm curious if this isn't a deeper issue presenting this error as a symptom. I'm getting the same error trying to run @google-cloud/text-to-speech. While the error appears to be in the error handling, I can run the exact same code with node and never get an error at all (my request goes through and I get my requested data back). Whereas even with the proposed fix, my request never gets a response (since there's still an underlying error that also only happens with bun).
This code is pulled almost exactly from the google TTS getting started page if you wanted a bit more:
const client = new textToSpeech.TextToSpeechClient({
keyFilename: 'key_location.json'
});
const request = {
input: {text: text},
// Select the language and SSML voice gender (optional)
voice: {languageCode: 'en-US', ssmlGender: 'NEUTRAL'},
// select the type of audio encoding
audioConfig: {audioEncoding: 'MP3'},
};
// Error
const [response] = await client.synthesizeSpeech(request);
// response handling
Using bun 1.0.2
Ubuntu 20.04.6 LTS x86_64
Also, if it's better to start a new issue I can do that.
Is it not because bun does not have an implementation of http2 yet ? It seems those google library use grpc protocol under the wood that needs http2.
This is happening because bun only calls Error.prepareStackTrace
when Error.captureStackTrace
is called. It needs to be called when accessing stack
.
input:
Error.prepareStackTrace = function (err, stack) {
return stack;
}
const error = new Error();
console.log(error.stack[0]);
node output: Callsite {}
bun output: <first character of stringified stack trace>
we are working on a fix for this.
Is it not because bun does not have an implementation of http2 yet ? It seems those google library use grpc protocol under the wood that needs http2.
This sounds like my underlying issue. This is my first project attempting bun, and I wasn't aware it was missing http2. I'm looking forward to it.
Thank you.
Caused by #4606
Using canary build 1.0.3+0b4a34bbd6e07096212774759ca6e8a7f3922e46, this issue is still coming up. Is it possible that the fix isn't in the canary build at this time and I am jumping the gun?
I'm using bun 1.0.3, and still getting the same error in the original message (originating from google-gax
).
Closed by #5802
Running the latest canary build of Bun and still experiencing the same issue.
node dist/myfile.ts
works just fine but bun run dist/myfile.ts
produces:
29 | const CALL_INVOCATION_TRANSFORMER_SYMBOL = Symbol();
30 | function isFunction(arg) {
31 | return typeof arg === 'function';
32 | }
33 | function getErrorStackString(error) {
34 | return error.stack.split('\n').slice(1).join('\n');
^
TypeError: undefined is not an object (evaluating 'error.stack.split')
at getErrorStackString (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client.js:34:11)
at onReceiveStatus (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client.js:356:40)
at onReceiveStatus (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:177)
at /Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:74
at processTicksAndRejections (:55:76)
Running the latest canary build of Bun and still experiencing the same issue.
node dist/myfile.ts
works just fine butbun run dist/myfile.ts
produces:29 | const CALL_INVOCATION_TRANSFORMER_SYMBOL = Symbol(); 30 | function isFunction(arg) { 31 | return typeof arg === 'function'; 32 | } 33 | function getErrorStackString(error) { 34 | return error.stack.split('\n').slice(1).join('\n'); ^ TypeError: undefined is not an object (evaluating 'error.stack.split') at getErrorStackString (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client.js:34:11) at onReceiveStatus (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client.js:356:40) at onReceiveStatus (/Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:177) at /Users/me/Developer/myapp/server/node_modules/google-gax/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:74 at processTicksAndRejections (:55:76)
Same here, from what I can tell this is an issue with most Google client libraries
I'm facing issue
Hi guys, after a bunch of poking around I found a way to start fixing this. I think it's an issue with the implementation of a Passthrough stream from node:stream
on Bun. I'm starting with a simple .doc("...").get()
on Firestore.
First, the error up above is one error, yes, but not the most important one. It's just a message about parsing the stack trace. If you "fix" that you're still going to get to the next step which is the actual Firestore issue.
Second, gRPC is a no-go for now. The error returned by trying to use Firestore with it is just Error
, no information - so you need to fallback to preferRest: true
on initializing Firestore. For that, it's only a parameter. (Also, this means no onSnapshot
features for now).
initializeFirestore(firebaseApp, { preferRest: true })
With that, you're going to get another error if you're trying to make a query.
error: Did not receive document for "[...]".
To solve it, you can go to /node_modules/@google-cloud/firestore/build/src/document-reader.js:90
and comment out stream.resume()
. This will make your Firestore queries start returning again.
Now, I took some rounds to arrive at that, and while messing with the code I commented the line and my request passed. Some remarks:
stream
on stream.resume()
is created by an utility in Firestore called requestStream
. You can find it at the relative path ./node_modules/@google-cloud/firestore/build/src/index.js:1386
. Inside of it, it calls another method _initializeStream
passing the original stream and returning a passthrough (https://nodejs.org/api/stream.html#class-streampassthrough)paused
, the original stream consumes the data from Firebase and it doesn't get buffered/saved, so when the code tries to stream.resume()
to consume, there's nothing there.I'm not too knowledgeable with either Bun's implementation of node:streams (and the Passthrough) or with streams themselves, so I might be missing something. To wrap the version of the firebase-admin
I'm using is "11.11.0"
.
TL;DR
preferRest
when initializing Firestore with
initializeFirestore(firebaseApp, { preferRest: true })
./node_modules/@google-cloud/firestore/build/src/document-reader.js:90
and comment out stream.resume()
.doc("...").get()
../node_modules/@google-cloud/firestore/build/src/reference.js:2018
and comment out backendStream.resume()
.collection("...").limit(3).get()
.Very early findings, I won't dive too much into this issue for now. If you go to ./node_modules/@google-cloud/firestore/build/src/index.js
and replace Firestore's logger with a simple console.log
you can see all your data returning on the request, it just doesn't get properly piped from the original stream (there called backendStream
) to the Passthrough stream.
As it was suggested a patch-package
above, I'm going to add one for my suggestion too. Here:
./patches/@google-cloud+firestore+6.8.0.patch
diff --git a/node_modules/@google-cloud/firestore/build/src/document-reader.js b/node_modules/@google-cloud/firestore/build/src/document-reader.js
index da32b26..a1cba10 100644
--- a/node_modules/@google-cloud/firestore/build/src/document-reader.js
+++ b/node_modules/@google-cloud/firestore/build/src/document-reader.js
@@ -88,7 +88,7 @@ class DocumentReader {
try {
const stream = await this.firestore.requestStream('batchGetDocuments',
/* bidirectional= */ false, request, requestTag);
- stream.resume();
+ // stream.resume();
for await (const response of stream) {
let snapshot;
if (response.found) {
diff --git a/node_modules/@google-cloud/firestore/build/src/reference.js b/node_modules/@google-cloud/firestore/build/src/reference.js
index 837dd09..e75a140 100644
--- a/node_modules/@google-cloud/firestore/build/src/reference.js
+++ b/node_modules/@google-cloud/firestore/build/src/reference.js
@@ -2015,7 +2015,7 @@ class Query {
backendStream.on('end', () => {
streamActive.resolve(/* active= */ false);
});
- backendStream.resume();
+ // backendStream.resume();
backendStream.pipe(stream);
} while (await streamActive.promise);
})
Also, obviously, this is not a reliable solution at all. Don't @ me if it breaks.
I'm also getting this issue. Looking forward to it being fixed. ๐
As it was suggested a
patch-package
above, I'm going to add one for my suggestion too. Here:
- File
./patches/@google-cloud+firestore+6.8.0.patch
diff --git a/node_modules/@google-cloud/firestore/build/src/document-reader.js b/node_modules/@google-cloud/firestore/build/src/document-reader.js index da32b26..a1cba10 100644 --- a/node_modules/@google-cloud/firestore/build/src/document-reader.js +++ b/node_modules/@google-cloud/firestore/build/src/document-reader.js @@ -88,7 +88,7 @@ class DocumentReader { try { const stream = await this.firestore.requestStream('batchGetDocuments', /* bidirectional= */ false, request, requestTag); - stream.resume(); + // stream.resume(); for await (const response of stream) { let snapshot; if (response.found) { diff --git a/node_modules/@google-cloud/firestore/build/src/reference.js b/node_modules/@google-cloud/firestore/build/src/reference.js index 837dd09..e75a140 100644 --- a/node_modules/@google-cloud/firestore/build/src/reference.js +++ b/node_modules/@google-cloud/firestore/build/src/reference.js @@ -2015,7 +2015,7 @@ class Query { backendStream.on('end', () => { streamActive.resolve(/* active= */ false); }); - backendStream.resume(); + // backendStream.resume(); backendStream.pipe(stream); } while (await streamActive.promise); })
Also, obviously, this is not a reliable solution at all. Don't @ me if it breaks.
Has anyone got this patch to work with the emulator? It seems to work for me with the remote db but not with the emulator
getting the same issue using @google-cloud/pubsub
Quick hack: just add Error.prototype.stack = '';
early in your program and it should all go away. ofc you wont get the stack, but you cant monkey patch stack trace generation anyway. So this just avoid the null pointers.
If youโre curious here is a nice rabbit hole to go down
@samuelslva-jlfs What you suggested causes I don't see any error, but the app still has a problem with querying db, it just freezes when requesting Firestore
Well I had this issue on a different gRPC GCP product, what happened was there was a issue building the stack strace, apparently GRPC has some fancy formatting on the error message and tries to access error.stack which causes the issue, after I injected the hack above the exception was thrown naturally which I then try-catched + console.log'ed it if you're here there is still an underlying exception that is being prevented from being thrown. @JeremJ if it seems to freeze try adding a low timeout to whatever query you're doing
With http2 included with 1.0.13, I don't get the same error anymore trying to run Firestore Admin SDK.
However, I can't seem to get anything when running stuff like await ref.doc(path).get()
. Seems like the Promise is pending forever. No error codes. Maybe this is an unrelated issue? Maybe because the whole http2 implementation doesn't seem to be out yet?
For me, it's a problem with cloud run.
Unfortunately patch-package
didn't help.
So I wrote my custom version of REST calls to do what the library is supposed to do. Recite from cloud task and auth documentation
async function getGCPAuthToken() {
const scope = 'https://www.googleapis.com/auth/cloud-platform'
const googleAuth = new GoogleAuth({
scopes: scope,
})
const { credential } =
await googleAuth.getApplicationDefault(googleServiceAccount)
// Create the impersonated credential.
const impersonatedCredentials = new Impersonated({
sourceClient: credential,
delegates: [],
targetPrincipal: googleServiceAccount.client_email,
targetScopes: [scope],
lifetime: 3600,
})
// Get the OAuth2 token.
// Once you've obtained the OAuth2 token, you can use it to make an authenticated call
// to the target audience.
const resp = await impersonatedCredentials.getAccessToken()
return resp.token
}
private scheduleTask = async (
queueName: string,
pathName: string,
payload: Record<string, string>,
triggerBy: Date,
) => {
const queryPath = JobScheduler.client.queuePath(
GCP_PROJECT_ID,
PROJECT_LOCATION_ID,
queueName,
)
const task = {
httpRequest: {
httpMethod: 'POST' as const,
url: `${SERVER_URL}${pathName}`,
body: Buffer.from(JSON.stringify(payload)).toString('base64'),
headers: {
'Content-Type': 'application/json',
},
},
scheduleTime: triggerBy,
}
const token = await getGCPAuthToken()
const url = `https://cloudtasks.googleapis.com/v2beta3/${queryPath}/tasks`
const options = {
method: 'POST',
headers: {
Authorization: `Bearer ${token}`,
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify({ task, responseView: 'FULL' }),
}
const data = await fetch(url, options)
return (await data.json()) as { name: string }
}
For ppl going down this path be mindful of the number of API calls to get auth token.
Ahh, good find @j1mmie . I was looking for something similar.
Ahh, good find @j1mmie . I was looking for something similar.
@treeder I spoke too soon actually. Higher up, the same solution is discussed, but it also requires a monkey patch.
With http2 included with 1.0.13, I don't get the same error anymore trying to run Firestore Admin SDK.
However, I can't seem to get anything when running stuff like
await ref.doc(path).get()
. Seems like the Promise is pending forever. No error codes. Maybe this is an unrelated issue? Maybe because the whole http2 implementation doesn't seem to be out yet?
Yes getting the exact issue here requests just never fulfill, tried the patches above but none of them work unfortunately, hopefully there is a workaround for this as I believe the whole firebase-admin package now can't be used.
Also getting the same issue. Forever pending promises, on firebase-admin
's firestore instance, but also on other libraries like mapbox-js-sdk
.
Maybe this would be worth opening a new issue.
Same is happening to @google-cloud/spanner
(tested only on 7.2.0
& bun 1.0.22
)
hanging promise without any immediate feedbacks until an error suddenly appears on the console after a while:
29 | const CALL_INVOCATION_TRANSFORMER_SYMBOL = Symbol();
30 | function isFunction(arg) {
31 | return typeof arg === 'function';
32 | }
33 | function getErrorStackString(error) {
34 | return error.stack.split('\n').slice(1).join('\n');
^
TypeError: undefined is not an object (evaluating 'error.stack.split')
at getErrorStackString (.../node_modules/@grpc/grpc-js/build/src/client.js:34:12)
at onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client.js:191:41)
at .../node_modules/@grpc/grpc-js/build/src/call-interface.js:78:17
at onReceiveStatus (.../node_modules/grpc-gcp/build/src/index.js:73:29)
at onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/call-interface.js:73:9)
at onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:138)
at onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:178)
at .../node_modules/@grpc/grpc-js/build/src/resolving-call.js:99:75
at processTicksAndRejections (:61:77)
Still seeing this on bun 1.0.25
and firebase-admin 12.0.0
Maybe someone should create a public repo with an example of all the issues with google-cloud and Firebase admin so that @Jarred-Sumner or someone else could take a look and maybe find solutions more easily
This particular issue is fixed in Bun v1.0.26, which releases in a few hours.
I just tried Firestore/Gcloud x Bun v1.0.26, but now I have an issue after the build at run time. The build works fine, but after I run start and I try to get a value from firestore, I get this error:
TypeError: exports.v1 is not a constructor
More details:
TypeError: exports.v1 is not a constructor
at ClientPool.clientFactory (file:///Users/userName/workspace/projectName/api/build/index.js:146559:20)
at ClientPool.acquire (file:///Users/userName/workspace/projectName/api/build/index.js:143860:31)
at ClientPool.run (file:///Users/userName/workspace/projectName/api/build/index.js:143919:27)
at file:///Users/userName/workspace/projectName/api/build/index.js:146913:26
at Firestore._retry (file:///Users/userName/workspace/projectName/api/build/index.js:146835:24)
at async DocumentReader.fetchDocuments (file:///Users/userName/workspace/projectName/api/build/index.js:143777:24)
at async DocumentReader.get (file:///Users/userName/workspace/projectName/api/build/index.js:143745:7)
at async file:///Users/userName/workspace/projectName/api/build/index.js:170237:24
at async resolveMiddleware (file:///Users/userName/workspace/projectName/api/build/index.js:163776:22)
at async callRecursive (file:///Users/userName/workspace/projectName/api/build/index.js:163799:25)
Caused by: Error
at Firestore.getAll (file:///Users/userName/workspace/projectName/api/build/index.js:146753:21)
at DocumentReference.get (file:///Users/userName/workspace/projectName/api/build/index.js:135809:30)
Details: it's slow, it takes around 10s to return this error
And also, in the build/index.js
, I can see:
exports.initializeFirestore = exports.getFirestore = exports.setLogFunction = exports.v1 = exports.WriteResult = exports.WriteBatch = exports.Transaction = exports.Timestamp = exports.QuerySnapshot = exports.QueryPartition = exports.QueryDocumentSnapshot = exports.Query = exports.GrpcStatus = exports.GeoPoint = exports.Firestore = exports.Filter = exports.FieldValue = exports.FieldPath = exports.DocumentSnapshot = exports.DocumentReference = exports.CollectionReference = exports.CollectionGroup = exports.BundleBuilder = exports.BulkWriter = undefined;
And when I do a console.log(exports.v1)
, I can see that exports.v1
is undefined
Bun v1.0.26 firebase-admin 12.0.0 Forever pending promises(. Unfortunately, the bug is still present.
Bun v1.0.26 firebase-admin 12.0.0 Forever pending promises(. Unfortunately, the bug is still present.
I just tried Firestore/Gcloud x Bun v1.0.26, but now I have an issue after the build at run time. The build works fine, but after I run start and I try to get a value from firestore, I get this error:
TypeError: exports.v1 is not a constructor
More details:
TypeError: exports.v1 is not a constructor at ClientPool.clientFactory (file:///Users/userName/workspace/projectName/api/build/index.js:146559:20) at ClientPool.acquire (file:///Users/userName/workspace/projectName/api/build/index.js:143860:31) at ClientPool.run (file:///Users/userName/workspace/projectName/api/build/index.js:143919:27) at file:///Users/userName/workspace/projectName/api/build/index.js:146913:26 at Firestore._retry (file:///Users/userName/workspace/projectName/api/build/index.js:146835:24) at async DocumentReader.fetchDocuments (file:///Users/userName/workspace/projectName/api/build/index.js:143777:24) at async DocumentReader.get (file:///Users/userName/workspace/projectName/api/build/index.js:143745:7) at async file:///Users/userName/workspace/projectName/api/build/index.js:170237:24 at async resolveMiddleware (file:///Users/userName/workspace/projectName/api/build/index.js:163776:22) at async callRecursive (file:///Users/userName/workspace/projectName/api/build/index.js:163799:25) Caused by: Error at Firestore.getAll (file:///Users/userName/workspace/projectName/api/build/index.js:146753:21) at DocumentReference.get (file:///Users/userName/workspace/projectName/api/build/index.js:135809:30)
Details: it's slow, it takes around 10s to return this error
And also, in the
build/index.js
, I can see:exports.initializeFirestore = exports.getFirestore = exports.setLogFunction = exports.v1 = exports.WriteResult = exports.WriteBatch = exports.Transaction = exports.Timestamp = exports.QuerySnapshot = exports.QueryPartition = exports.QueryDocumentSnapshot = exports.Query = exports.GrpcStatus = exports.GeoPoint = exports.Firestore = exports.Filter = exports.FieldValue = exports.FieldPath = exports.DocumentSnapshot = exports.DocumentReference = exports.CollectionReference = exports.CollectionGroup = exports.BundleBuilder = exports.BulkWriter = undefined;
And when I do a
console.log(exports.v1)
, I can see thatexports.v1
isundefined
Both of these are different errors. This specific issue (about error.stack.split
) has been fixed. This issue thread refers to an error while printing an error.
The underlying cause is HTTP2 client reliability issues, which I think might be fixed in the canary build due to recent changes by @cirospaciari. Can you try bun upgrade --canary
?
I tried to upgrade to Canary code:
const app = initializeApp({
credential: credential.cert(serviceAccount),
});
const fireBase = getFirestore(app);
const user = await fireBase.collection("*").doc("*").get();
console.log(user);
This code never reaches console.log, no errors are displayed in the console
I tried to upgrade to Canary code:
const app = initializeApp({ credential: credential.cert(serviceAccount), }); const fireBase = getFirestore(app); const user = await fireBase.collection("*").doc("*").get(); console.log(user);
This code never reaches console.log, no errors are displayed in the console
I have upgrade to Canary too (1.0.27) and got this problem in debugger
And now think it's an issue with bun's h2 parser
This is the headers sent to @grpc/grpc-js, same in node
What version of Bun is running?
1.0.0+822a00c4d508b54f650933a73ca5f4a3af9a7983
What platform is your computer?
Darwin 22.6.0 arm64 arm
What steps can reproduce the bug?
File: firebase.ts
Run
What is the expected behavior?
Connect to firestore
What do you see instead?
Additional information
What am I missing?