Unable to consume AI Provider using Bearer Token as authentication method.

[DavidAcero]

What are you trying to achieve?

Consume AI provider through Bearer token as authentication method instead of API_KEY.

What do you get instead?

Access denied error since the AI provider was set to just allow authentication through Bearer token instead of API_KEY.

Provide test source code if related

Not Applied

Details

• CodeceptJS version: 3.6.3
• NodeJS Version: 14.21.1
• Operating System: MacOS
• Configuration file: N.A.

[kobenguyent]

Interesting! Could you please give us more info regarding the AI section in Codeceptjs conf? That'll help us understand more the context.

[DavidAcero]

Interesting! Could you please give us more info regarding the AI section in Codeceptjs conf? That'll help us understand more the context.

Sure thing

[kobenguyent]

maybe this help @DavidAcero

      request: async (messages) => {
        const {getBearerTokenProvider, DefaultAzureCredential} = require("@azure/identity");
        const AzureOpenAI = require("openai");

        const scope = "https://cognitiveservices.azure.com/.default";
        const azureADTokenProvider = getBearerTokenProvider(new DefaultAzureCredential(), scope);
        const deployment = "YOUR_DEPLOYMENT_NAME";
        const apiVersion = "2024-04-01-preview";
        const client = new AzureOpenAI({ azureADTokenProvider, deployment, apiVersion });
        const result = await client.chat.completions.create({
          messages,
          model: 'gpt-3.5-turbo', // your preferred model
        });

        return result.choices[0]?.message?.content;
      }

it worked with my test

  Github login
    I am on page "https://github.com"
    I ask for page object "login page"
⠙  Processing AI request...azure:identity:info EnvironmentCredential => Found the following environment variables: 
azure:identity:info WorkloadIdentityCredential => Found the following environment variables: 
azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
⠸  Processing AI request...[
  'const { I } = inject();\n' +
    '\n' +
    'module.exports = {\n' +
    '\n' +
    '// setting locators\n' +
    "element1: locate().withText('Skip to content'),\n" +
    "element2: locate('a').withAttr({class: 'show-on-focus js-skip-to-content'}),\n" +
    "element3: locate('a').withText('Skip to content').inside('div.js-header-wrapper'),\n" +
    "element4: locate('a').withAttr({class: 'Link--inTextBlock Link--outlineOffset no-wrap'}),\n" +
    '\n' +
    '// setting methods\n' +
    'doSomethingOnPage(params) {\n' +
    '// ...\n' +
    '},\n' +
    '}'
]
----- Generated PageObject ----
const {
  I
} = inject();

module.exports = {

  // setting locators
  element1: locate().withText('Skip to content'),
  element2: locate('a').withAttr({
    class: 'show-on-focus js-skip-to-content'
  }),
  element3: locate('a').withText('Skip to content').inside('div.js-header-wrapper'),
  element4: locate('a').withAttr({
    class: 'Link--inTextBlock Link--outlineOffset no-wrap'
  }),

  // setting methods
  doSomethingOnPage(params) {
    // ...
  },
}
-------------------------------
Page object for login page is saved to /Users/t/Desktop/projects/codecept-ai-demo/output/login pagePage-1720604954840.js
Page object registered for this session as `page` variable
Use `=>page.methodName()` in shell to run methods of page object
Use `click(page.locatorName)` to check locators of page object
  ✔ OK in 4655ms

  OK  | 1 passed   // 5s
AI assistant took 3s and used ~10K input tokens. Tokens limit: 1000K

[DavidAcero]

Hi @kobenguyent , I tried to use your suggestion on both ways but I'm getting different error messages:

    request: async (messages) => {
     // Using @azure/openai library
      const AzureOpenAI = require("@azure/openai");
      const azureADTokenProvider = {
        token_type: "Bearer",
        expires_in: 9999,
        ext_expires_in: 9999,
        access_token: "MY_ACCESS_TOKEN"
      };
      const deployment = "MY_DEPLOYMENT_NAME";
      const apiVersion = "2024-04-01-preview";
      const client = new AzureOpenAI({ azureADTokenProvider, deployment, apiVersion });
      const result = await client.chat.completions.create({
        messages,
        model: 'gpt-3.5-turbo', // your preferred model
      });
      return result.choices[0]?.message?.content;
    },

But the error I was getting is the following:

⠋ Processing AI request...
AI service error: AzureOpenAI is not a constructor

Then I switch the library to "openai" but getting a different error:

    request: async (messages) => {
      const AzureOpenAI = require("openai");
      const azureADTokenProvider = {
        token_type: "Bearer",
        expires_in: 9999,
        ext_expires_in: 9999,
        access_token: "MY_ACCESS_TOKEN"
      };
      const deployment = "MY_DEPLOYMENT_NAME";
      const apiVersion = "2024-04-01-preview";
      const client = new AzureOpenAI({ azureADTokenProvider, deployment, apiVersion });
      const result = await client.chat.completions.create({
        messages,
        model: 'gpt-3.5-turbo', // your preferred model
      });
      return result.choices[0]?.message?.content;
    },

The new error to be received is the following:

⠋ Processing AI request...
AI service error: The OPENAI_API_KEY environment variable is missing or empty; either provide it, or instantiate the OpenAI client with an apiKey option, like new OpenAI({ apiKey: 'My API Key' }).

[kobenguyent]

all right, I hope this would resolve the issue now.

Firstly, creating a function to get bearer token

function getAzureBearerToken() {
  const axios = require("axios").default;

  const form = new FormData();
  form.append("grant_type", "client_credentials");
  form.append("client_id", process.env.CLIENT_ID);
  form.append("client_secret", process.env.CLIENT_SECRET);

  const options = {
    method: 'POST',
    url: `https://login.microsoftonline.com/${process.env.RESOURCE_ID}/oauth2/token`,
    data: form
  };

  return axios.request(options).then(function (response) {
    return response.data
  }).catch(function (error) {
    console.error(error);
  });
}

Then calling it on your setup

...
    AI: {
      request: async (messages) => {
        const azureADTokenProvider = getAzureBearerToken()

        const deployment = process.env. DEPLOYMENT_NAME;
        const apiVersion = "2024-04-01-preview";
        const client = new AzureOpenAI({ azureADTokenProvider, deployment, apiVersion });
        const result = await client.chat.completions.create({
          messages,
          model: 'gpt-3.5-turbo',
        });

        return result.choices[0]?.message?.content;
      }
    }
...

And you're good to go

***************************************
nodeInfo:  18.19.0
osInfo:  macOS 14.4
cpuInfo:  (8) x64 Apple M1 Pro
chromeInfo:  126.0.6478.127
edgeInfo:  126.0.2592.102
firefoxInfo:  undefined
safariInfo:  17.4
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v3.6.5-beta.5 #StandWithUkraine
Using test root "/Users/t/Desktop/projects/codecept-ai-demo"
Helpers: Playwright, AI
Plugins: screenshotOnFail, heal

ai --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  Github login
    I am on page "https://github.com"
    I ask for page object "login page"
⠼  Processing AI request...[
  'const { I } = inject();\n' +
    '\n' +
    'module.exports = {\n' +
    '\n' +
    '// setting locators\n' +
    "linkProduct: locate('a').withText('Product'),\n" +
    "linkSolutions: locate('a').withText('Solutions'),\n" +
    "linkResources: locate('a').withText('Resources'),\n" +
    "linkOpenSource: locate('a').withText('Open Source'),\n" +
    "linkEnterprise: locate('a').withText('Enterprise'),\n" +
    "linkPricing: locate('a').withText('Pricing'),\n" +
    "linkFeatures: locate('a').withText('All features'),\n" +
    "inputEmail: locate('input').withAttr({ name: 'user_email' }),\n" +
    "buttonSignIn: locate('a').withText('Sign in'),\n" +
    "buttonSignUp: locate('a').withText('Sign up'),\n" +
    '\n' +
    '// seting methods\n' +
    'doSomethingOnPage(params) {\n' +
    '// ...\n' +
    '},\n' +
    '}'
]
----- Generated PageObject ----
const {
  I
} = inject();

module.exports = {

  // setting locators
  linkProduct: locate('a').withText('Product'),
  linkSolutions: locate('a').withText('Solutions'),
  linkResources: locate('a').withText('Resources'),
  linkOpenSource: locate('a').withText('Open Source'),
  linkEnterprise: locate('a').withText('Enterprise'),
  linkPricing: locate('a').withText('Pricing'),
  linkFeatures: locate('a').withText('All features'),
  inputEmail: locate('input').withAttr({
    name: 'user_email'
  }),
  buttonSignIn: locate('a').withText('Sign in'),
  buttonSignUp: locate('a').withText('Sign up'),

  // seting methods
  doSomethingOnPage(params) {
    // ...
  },
}
-------------------------------
Page object for login page is saved to /Users/t/Desktop/projects/codecept-ai-demo/output/login pagePage-1720761010570.js
Page object registered for this session as `page` variable
Use `=>page.methodName()` in shell to run methods of page object
Use `click(page.locatorName)` to check locators of page object
  ✔ OK in 5608ms

  OK  | 1 passed   // 6s
AI assistant took 5s and used ~10K input tokens. Tokens limit: 1000K

[DavidAcero]

@kobenguyent , sorry for the delay, I tried to follow the provided advices, however I'm still getting stuck with the same error, token is retrieved successfully, however at the moment I try to use AI it is requiring me an API-KEY

[kobenguyent]

hey @DavidAcero I could replicate the same issue, cause I forgot to remove the OPEN_API_KEY

May you try another approach?

      request: async (messages) => {
        try {
          const { OpenAIClient} = require("@azure/openai");
          const { DefaultAzureCredential } = require("@azure/identity");

          const endpoint = process.env.API_ENDPOINT;
          const client = new OpenAIClient(endpoint, new DefaultAzureCredential());
          const deploymentId = process.env.DEPLOYMENT_ID;

          const result = await client.getCompletions(deploymentId, {
            prompt: messages,
            model: 'gpt-3.5-turbo'
          });

          return result.choices[0]?.text;
        } catch (error) {
          console.error("Error calling API:", error);
          throw error;
        }
      }

Don't forget to set those env vars

azure:identity:info EnvironmentCredential => Found the following environment variables: AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET

I could not get the response due to the azure setup as I don't have much knowledge on this. But you could try, I think the setup from your end would be done.

[DavidAcero]

@kobenguyent , thanks a lot, it seems I'm able to send the request to Azure OpenAI. However apparently the model used does not support completion. Do you happen to know which operation CodeceptJS is using on the AI task, or, even better, which azure open ai models are supported on CodeceptJS?

For now I think the Bearer Token is successfully being consume.

[kobenguyent]

@DavidAcero the model is defined by model deployments on azure open ai. I guess you could find it out and pass the correct model there. Technically, codeceptjs won't control which model is used yet explicitly defined by users.