You need to pass auth instance to gRPC-fallback client. Use OAuth2Client from google-auth-library

[gabrielgouv]

I dont know if it is a bug or not, but I'm trying to use the Dialogflow lib in a existing Vue project and I can't use because I always receive this error:

You need to pass auth instance to gRPC-fallback client. Use OAuth2Client from google-auth-library

I created a new Node.js project from scratch and copy and paste the exactly same code (and key) that I used in my Vue project and it works fine, but in my Vue project it does not work.

I tried to use the google-auth-library to authenticate, but no success.

This is my code:

      async detectTextIntent(text, sessionId, languageCode) {

        const sessionClient = new dialogflow.SessionsClient(this.config)

        const sessionPath = sessionClient.sessionPath(this.config.projectId, sessionId);

        const request = {
            session: sessionPath,
            queryInput: {
                text: {
                    text: text,
                    languageCode,
                },
            },
        };

        return await sessionClient.detectIntent(request);

    }

The config object is the credentials (I've tried using file key path and ENV variable also):

    const config = {
        projectId: "myprojectid",
        credentials: {
          private_key: "-----BEGIN PRIVATE KEY-----\n[...]\n-----END PRIVATE KEY-----\n",
          client_email: "xxxxx-xxxxxxx-xxxxxxxx@xxxxx-xxxxx-xxxx.iam.gserviceaccount.com"
        }
      }

I dont know if can be a conflict with an existing lib.

[bcoe]

@gabrielgouv what approach does Vue use for bundling JavaScript for client side libraries? At this time most of our libraries don't have officially have browser support (see the discussion here.

We are however making strides towards adding support ... in the short term, it might be better to write a server layer that your view component interacts with, that in turn calls the dialogflow API.

[alexander-fenster]

@gabrielgouv As @bcoe said, the browser support is half way there. It's already in one of the middleware libraries (google-gax), it's why you see the warning, but it's not yet in the library code- I'm going to merge some more changes this week and it will likely start working (in a limited fashion, with no streaming available - more on that in the release notes). You'll definitely need to change your auth workflow though.

Let's leave this open for now and I'll get back to this issue when more library changes are merged. But if you need to have a complete support with service accounts, consider using the backend server as @bcoe suggested.

[gabrielgouv]

@bcoe, @alexander-fenster I understand, at the moment I chose to use the REST API, since I didn't have much time. Via REST I can communicate with Dialogflow normally.

[bcoe]

@gabrielgouv thanks for your patience :+1: we intend to make browser support better over time.

[RyanF18]

@bcoe Hello! I am having the same issue...

You need to pass auth instance to use gRPC-fallback client in browser. Use OAuth2Client from google-auth-library.

I am referencing the Cloud AutoML: Node.js Client documentation. The error arises when I create the client for prediction service...'const client = new automl.PredictionServiceClient();' I would appreciate any guidance on how to resolve this issue.

Referenced Documentation: https://googleapis.dev/nodejs/automl/latest/index.html

[alexander-fenster]

@RyanF18 Are you trying to use the library in the browser, Electron app, or something like that?

[RyanF18]

@alexander-fenster I just got it working but thank you for the response!

[ushypocrites]

hi is there any resolution to the first error by gabrielgou? i have the same issue and not sure what to do. do you have a recipe to get around the issue?

[ushypocrites]

@gabrielgouv what approach does Vue use for bundling JavaScript for client side libraries? At this time most of our libraries don't have officially have browser support (see the discussion here.

We are however making strides towards adding support ... in the short term, it might be better to write a server layer that your view component interacts with, that in turn calls the dialogflow API.

The link doesn't lead to anymore, can you post a recipe on how to make dialogflow v2 work for browsers? Thanks.

[alexander-fenster]

Hey folks @ushypocrites @gabrielgouv,

TL;DR: this library is supposed to be used from a server-side Node.js application, not from any front-end environment such as a browser, Electron app, React, (name your front-end framework here). If you just run the code by plain regular Node.js, it will work.

Having said that - we do have experimental support for a browser use case starting from the latest version, which is 0.11.0. It's experimental (just implemented) and not really documented yet. You can try using it though. To do that, you need to pass an authenticated instance of OAuth2Client (from google-auth-library) as an auth parameter of the client constructor:

const auth = require('google-auth-library');
const oauth2client = new auth.OAuth2Client(client_id, client_secret, callback_uri);
const authUrl = oauth2client.generateAuthUrl({
  access_type: 'offline',
  scope: [    // scopes for Dialogflow
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/dialogflow'
  ]
});
// redirect user to authUrl and wait for them coming back to callback_uri

// in callback_uri handler, get the auth code from query string and obtain a token:
const tokenResponse = await oauth2client.getToken(code);
oauth2client.setCredentials(tokenResponse.tokens);

// now use this oauth2client!
const sessionClient = new dialogflow.SessionsClient({ auth: oauth2client }); // <-- auth passed here

Since it all was baked just recently, we don't have any published documentation on that. If you folks explain your use cases (at least, what kind of environment are you targeting - is it Electron, or a regular browser, or whatever else), I might be able to provide more specific example.

But - back to TL;DR - the officially supported and easiest way to use this client library is to use it with Node.js on the server side.

[ushypocrites]

Hey folks @ushypocrites @gabrielgouv,

TL;DR: this library is supposed to be used from a server-side Node.js application, not from any front-end environment such as a browser, Electron app, React, (name your front-end framework here). If you just run the code by plain regular Node.js, it will work.

Having said that - we do have experimental support for a browser use case starting from the latest version, which is 0.11.0. It's experimental (just implemented) and not really documented yet. You can try using it though. To do that, you need to pass an authenticated instance of OAuth2Client (from google-auth-library) as an auth parameter of the client constructor:

const auth = require('google-auth-library');
const oauth2client = new auth.OAuth2Client(client_id, client_secret, callback_uri);
const authUrl = oauth2client.generateAuthUrl({
  access_type: 'offline',
  scope: [    // scopes for Dialogflow
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/dialogflow'
  ]
});
// redirect user to authUrl and wait for them coming back to callback_uri

// in callback_uri handler, get the auth code from query string and obtain a token:
const tokenResponse = await oauth2client.getToken(code);
oauth2client.setCredentials(tokenResponse.tokens);

// now use this oauth2client!
const sessionClient = new dialogflow.SessionsClient({ auth: oauth2client }); // <-- auth passed here

Since it all was baked just recently, we don't have any published documentation on that. If you folks explain your use cases (at least, what kind of environment are you targeting - is it Electron, or a regular browser, or whatever else), I might be able to provide more specific example.

But - back to TL;DR - the officially supported and easiest way to use this client library is to use it with Node.js on the server side.

@alexander-fenster Thanks so much for this! I will give it a try. The use case is a chat window in a react js website (not app) on a regular browser. I do see a few websites that have a chat UI and use dialogflow so i am wondering how they are able to do this.

[alexander-fenster]

@ushypocrites There are at least five options how they can do it!

1. As above (I bet nobody uses it this way as for now since it was released just yesterday and not yet documented, other than in this issue :) )

2. They might use server side (backend) script. That's probably the easiest option.

3. They might use our other library, googleapis. It supports dialogflow and is webpackable as described here. It's a little bit less fancy than this one though.

4. They might use Google JavaScript API using the discovery document.

5. Finally, they might be using JSON requests directly (again, this link).

My personal opinion here is that the easiest way is probably to have a backend (option 2), which probably gives you a lot of extra benefits compared to in-browser solution. Out of other four solutions, I'd prefer 1, 3, 4, 5, in this order.

[ushypocrites]

@ushypocrites There are at least five options how they can do it!

1. As above (I bet nobody uses it this way as for now since it was released just yesterday and not yet documented, other than in this issue :) )
2. They might use server side (backend) script. That's probably the easiest option.
3. They might use our other library, googleapis. It supports dialogflow and is webpackable as described here. It's a little bit less fancy than this one though.
4. They might use Google JavaScript API using the discovery document.
5. Finally, they might be using JSON requests directly (again, this link).

My personal opinion here is that the easiest way is probably to have a backend (option 2), which probably gives you a lot of extra benefits compared to in-browser solution. Out of other four solutions, I'd prefer 1, 3, 4, 5, in this order.

Thanks so much! That's very useful. I am now considering 2) and perhaps using firebase. Do you have any comments about firebase? does the conversation context and intent get lost if there are long pauses between requests?

[gabrielgouv]

Hey folks @ushypocrites @gabrielgouv,

TL;DR: this library is supposed to be used from a server-side Node.js application, not from any front-end environment such as a browser, Electron app, React, (name your front-end framework here). If you just run the code by plain regular Node.js, it will work.

Having said that - we do have experimental support for a browser use case starting from the latest version, which is 0.11.0. It's experimental (just implemented) and not really documented yet. You can try using it though. To do that, you need to pass an authenticated instance of OAuth2Client (from google-auth-library) as an auth parameter of the client constructor:

const auth = require('google-auth-library');
const oauth2client = new auth.OAuth2Client(client_id, client_secret, callback_uri);
const authUrl = oauth2client.generateAuthUrl({
  access_type: 'offline',
  scope: [    // scopes for Dialogflow
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/dialogflow'
  ]
});
// redirect user to authUrl and wait for them coming back to callback_uri

// in callback_uri handler, get the auth code from query string and obtain a token:
const tokenResponse = await oauth2client.getToken(code);
oauth2client.setCredentials(tokenResponse.tokens);

// now use this oauth2client!
const sessionClient = new dialogflow.SessionsClient({ auth: oauth2client }); // <-- auth passed here

Since it all was baked just recently, we don't have any published documentation on that. If you folks explain your use cases (at least, what kind of environment are you targeting - is it Electron, or a regular browser, or whatever else), I might be able to provide more specific example.

But - back to TL;DR - the officially supported and easiest way to use this client library is to use it with Node.js on the server side.

Nice. I will try to use later. Im currently transitioning my current front end code (which used the REST API) to my back end.

[alexander-fenster]

@ushypocrites Firebase (cloud functions) is totally fine, and this library will work in cloud functions.

As for losing the context and intent because of the long pause, I unfortunately don't know the details of the specific API (I do work on client libraries in general, not the particular Dialogflow API). This doc has some information about context lifespans though :)

[ushypocrites]

@ushypocrites Firebase (cloud functions) is totally fine, and this library will work in cloud functions.

As for losing the context and intent because of the long pause, I unfortunately don't know the details of the specific API (I do work on client libraries in general, not the particular Dialogflow API). This doc has some information about context lifespans though :)

Thank you!

[urosran]

@alexander-fenster thank you for explaining this in such depth! Very nice to understand why something doesn't work after a day of trying to make it work.

It may be useful to put a note in the docs that the support for browsers is not available :)

[alexander-fenster]

@urosran ...it now works with browsers :) Unfortunately, no docs and samples yet, so we don't advertise it until we have everything ready. Long story short, as I wrote above, create an instance of OAuth2Client from google-auth-library, pass it to the client constructor, and it will work in browser.

If you tell me a little bit more about your use case (if it's an Electron app, or web page, or something else) I might be able to provide some code hints for you.

[naresht1710]

I had similar issue connecting Firestore, im trying to connect with service account from client side(browser) runs on nodejs, here is the code snippet

const admin = require('firebase-admin');

var serviceAccount = require('./../key.json');

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount)
});

var db = admin.firestore();

db.collection('xxxxx').get()
            .then((snapshot) => {

            })
            .catch((err) => {
                console.log('Error getting documents', err);
            });

i have got below error Uncaught (in promise) Error: {"clientConfig":{},"port":443,"servicePath":"firestore.googleapis.com","credentials":{"private_key":"-----BEGIN PRIVATE KEY-----\nMIIEvgIB......X\n-----END PRIVATE KEY-----\n","client_email":"xxxxiam.gserviceaccount.com"},"projectId":"xxxxx","firebaseVersion":"8.6.0","libName":"gccl","libVersion":"2.4.0 fire/8.6.0","scopes":["https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/datastore"]}You need to pass auth instance to use gRPC-fallback client in browser. Use OAuth2Client from google-auth-library.

couldn't figured out the reason behind, can you please help me.

[ajimen94]

Hi @alexander-fenster, thanks so much for your help so far. I am trying to get dialogflow to work on my react app. I was able to authenticate thanks to the steps you provided above, however, I come across an issue with CORS shown below.

I have tried setting Access-Control-Allow-Credentials to true within gaxios.

My detectIntent function is shown below:

[alexander-fenster]

@ajimen94 From what I understand about CORS (and Chrome - I suspect you might be using Chrome/Chromium, right?) it might not play well with CORS when your origin is http://localhost:8080. Any chance you can try using some real domain name? (even a fake one, through /etc/hosts or a local DNS server, should be better)

You can also try this to disable CORS in gaxios (I haven't tried it but might work):

const gaxios = require('gaxios');
gaxios.instance.defaults = {
  mode: 'no-cors',
};

[Zanyy]

Hello Guys, I have the same issue with Vision API using Electron. Code looks like this:

const vision = require('@google-cloud/vision');
const visionClient = new vision.ImageAnnotatorClient({
  keyFilename: 'Keyfile.json',
});

Any Suggestions ?

[lauchness]

@alexander-fenster I'm making some headway with your original post, but I've run into an issue trying to run the dialogflow.SessionsClient constructor, I'm getting the error Unhandled Rejection (TypeError): Cannot read property 'cloud' of undefined. Any ideas why this might be happening?

[alexander-fenster]

@lauchness I suspect it may be failing here:

https://github.com/googleapis/nodejs-dialogflow/blob/44cd81cc6404c926f007a4868728a514116aed81/src/v2/sessions_client.js#L159-L162

Can you double check that opts.fallback is true there? It should follow the lookupService route, not try to access protos.google.cloud (this is likely where it fails for you).

[lauchness]

@alexander-fenster Yep, that's the spot. I'm finding opts.fallback is undefined at this point. I'll keep looking into it, but do you have any insights as to why this might be the case for me?

[alexander-fenster]

@lauchness Then just pass { fallback: true } to the dialogflow.SessionsClient constructor. Soon (within the next update) it will start doing much better job autodetecting if a fallback mode is needed, but for now, passing an option should be enough.

Also, make sure you pass an auth instance as described in my post above.

[lauchness]

@alexander-fenster That's great! I also noticed this line:

https://github.com/googleapis/nodejs-dialogflow/blob/44cd81cc6404c926f007a4868728a514116aed81/src/v2/sessions_client.js#L64 And found that setting global.isBrowser = true, just before the constructor also fixes the bug. Though I don't feel like this is the best choice.

I'll go with your suggestion for now.

Is there any way I can contribute and help out here?

[alexander-fenster]

The original idea behind this global.isBrother thingy was that webpack (or similar bundler) will follow "browser": key in package.json, which will point to src/browser.js, which will set this global.isBrowser to true. This plan is half way done: src/browser.js is here, but "browser": key in package.json is not :)

There is no need to contribute right now because the code of this library (and most of the other @google-cloud/ libraries) is auto-generated. We are currently in process of moving them all to the new code generator that will handle it differently (instead of looking at global.isBrowser, we'll just check if window is defined, which is much easier and more reliable).

So, if passing fallback works for you, leave it this way. I expect that the problem will be fixed in the next release.

[lauchness]

@alexander-fenster Thanks! Also, passing fallback, isn't working, but setting global.isBrowser is. This is just for a personal experiment for now, so I'm not concerned if this will be improved soon.

[lauchness]

@alexander-fenster After getting the OAuth2 working, I'm now also encountering CORS issues with the dialogflow.SessionClient. Seems the response doesn't contain the 'Access-Control-Allow-Origin' header. I'm assuming this is because the library is intended to be used on a Node.js server. Is that correct? Are there any workarounds for using this on a client app? I'm currently trying it out in a React app.

[alexander-fenster]

CORS headers should be there. I did not try it for this particular library, but did not have any issues with them for other libraries (@google-cloud/language, @google-cloud/text-to-speech, etc.) Are you running it on localhost and with Chrome/Chromium? If so, make sure you're using a fully qualified domain name (even a fake one, pointing to 127.0.0.1, will work).

[lauchness]

@alexander-fenster So, I am running on localhost with Chrome and I've gone through the process of setting up my /etc/hosts file, so now my app is running at http://test-dialogflow.com:3000/. I've got the OAuth2 all working nicely, but my response from the sessionClient.detectIntent is consistently a 403 on the pre-flight request. (See screenshot here)

I've tested in both postman and my console, and am able to make the request successfully, but in both instances there are no "Access-Control-Allow-Origin" headers present on the response, which is consistent with my experience in the browser.

Seems like this an opportunity to open a separate ticket.

[alexander-fenster]

Yes, please feel free to open a separate issue. I will try to repro this with Dialogflow API (as I said, I got it working with others, but things might be different here) and we'll figure out what to do from there. If other folks in this thread got this working in Chrome, please feel free to chime in :)

[davidius]

@alexander-fenster what does code refer to in this line: const tokenResponse = await oauth2client.getToken(code);? It doesn't seem to be declared earlier in the code. Is this something I can get from the console?

[lauchness]

@davidius

@alexander-fenster what does code refer to in this line: const tokenResponse = await oauth2client.getToken(code);? It doesn't seem to be declared earlier in the code. Is this something I can get from the console?

The line above has a comment which mentions get the auth code from query string. You're supposed to retrieve the code which is sent back in the URL from the OAuth2 response.

[alexander-fenster]

@davidius The code refers to the one-time authentication code provided by the authentication server, which you later exchange to an OAuth2 token. It will be in the querystring parameter, as stated in the comment above that line:

// in callback_uri handler, get the auth code from query string and obtain a token:

In the real code, it will depend on how you actually receive this callback. In a pure browser usage, OAuth2 server will redirect your browser to callback_uri. If your website is example.com, it's e.g. https://example.com/cb and on that page you'll be able to do

function getAuthCode() {
  const code = new URL(window.location).searchParams.get('code');
  return code;
}

If you are in an Electron app or in anything similar, you can temporary launch the local webserver on some port, e.g. 3000, set callback_uri to http://localhost:3000/cb, listen to that one request that the authenticating browser will send when redirected to a callback URI, and get the code in the webserver request handler. E.g. for express, it will be

function getAuthCode() {
  return new Promise( (resolve, reject) => {
    const app = express();
    const server = app.listen(3000);
    app.get('/cb', (req, res) => {
      resolve(req.query.code);
      server.close();
    });
  });
}

[davidius]

makes sense, thanks very much!

[b-loved-dreamer]

Closing this issue. A solution seems to have been provided for this issue. If you disagree, please feel free to re-open this issue.

[markwindsorr]

is there documentation on this yet?