Problem after succesfully deploying the service to CF

[oliver-d148]

Hey guys,

I have some issues with the deployment of the service to CAP. This is an excerpt from the console: ... Deploying in org "afd72939trial" and space "dev" Detected MTA schema version: "3" Detected deployed MTA with ID "smart-converter-api" and version "1.0.0" Detected new MTA version: "1.0.0" Deployed MTA version: "1.0.0" Processing service "smart-converter-api-auth"... Setting service "smart-converter-api-auth" parameters from "xs-security.json" Updating service "smart-converter-api-auth"... Processing service "smart-converter-api-db"... Processing service "smart-converter-aicore-dest"... Updating service "smart-converter-aicore-dest"... 1 of 1 done 1 of 1 done Updating application "smart-converter-api-srv"... Application "smart-converter-api-srv" attributes are not modified and will not be updated Service instance "smart-converter-api-db" will not be rebound to application "smart-converter-api-srv" because the binding parameters are not modified Updating application "smart-converter-api-db-deployer"... Application "smart-converter-api-db-deployer" attributes are not modified and will not be updated Service instance "smart-converter-api-db" will not be rebound to application "smart-converter-api-db-deployer" because the binding parameters are not modified Uploading application "smart-converter-api-db-deployer"... Content of application "smart-converter-api-db-deployer" is not changed - upload will be skipped. Scaling application "smart-converter-api-db-deployer" to "1" instances... Executing task "deploy" on application "smart-converter-api-db-deployer"... Unbinding service instance "smart-converter-api-auth" from application "smart-converter-api-srv"... Unbinding service instance "smart-converter-aicore-dest" from application "smart-converter-api-srv"... Binding service instance "smart-converter-api-auth" to application "smart-converter-api-srv"... Binding service instance "smart-converter-aicore-dest" to application "smart-converter-api-srv"... Uploading application "smart-converter-api-srv"... Content of application "smart-converter-api-srv" is not changed - upload will be skipped. Stopping application "smart-converter-api-srv"... Starting application "smart-converter-api-srv"... Application "smart-converter-api-srv" started and available at "afd72939trial-dev-smart-converter-api-srv.cfapps.us10-001.hana.ondemand.com" Skipping deletion of services, because the command line option "--delete-services" is not specified. Process finished. Use "cf dmol -i ec77401a-0c36-11ef-8673-eeee0a925404" to download the logs of the process.

As you can see the process succesfully finished. But it doesn't work like the prompt "cds-ts watch --profile hybrid". By testing the service with the prompt above the server.ts is switched in front of it and with REST POST/GET Request with the api-key it worked:

$cds-ts watch --profile hybrid

cds serve all --with-mocks --in-memory? --profile hybrid live reload enabled for browsers resolving cloud service bindings... Getting service key smart-converter-aicore-dest-key Getting service key smart-converter-api-db-key bound destinations to Cloud Foundry managed service smart-converter-aicore-dest:smart-converter-aicore-dest-key bound db to Cloud Foundry managed service smart-converter-api-db:smart-converter-api-db-key

    ___________________________

[cds] - loading server from { file: 'srv/server.ts' } [cds] - loaded model from 3 file(s):

srv/converter.cds db/carbon.cds node_modules/@sap/cds/common.cds

[cds] - connect using bindings from: { registry: '~/.cds-services.json' } [cds] - connect to db > hana { database_id: '3a4093dd-8241-477a-9b31-fb29343d36cc', host: '3a4093dd-8241-477a-9b31-fb29343d36cc.hana.trial-us10.hanacloud.ondemand.com', port: '443', driver: 'com.sap.db.jdbc.Driver', url: 'jdbc:sap://3a4093dd-8241-477a-9b31-fb29343d36cc.hana.trial-us10.hanacloud.ondemand.com:443?encrypt=true&validateCertificate=true&currentschema=35F1C861A5FF46EB99875A694F6CE042', schema: '35F1C861A5FF46EB99875A694F6CE042', certificate: '...', hdi_user: '35F1C861A5FF46EB99875A694F6CE042_BWYLJUHOYTU32CFR950BXEUZF_DT', hdi_password: '...', user: '35F1C861A5FF46EB99875A694F6CE042_BWYLJUHOYTU32CFR950BXEUZF_RT', password: '...' } [cds] - No XSUAA instance bound to application, but "xsuaa" configured. This is NOT recommended in production!

[cds] - serving ConverterService { path: '/converter', impl: 'srv/converter.ts' }

[cds] - server listening on { url: 'http://localhost:4004' } [cds] - launched at 5/7/2024, 6:23:25 AM, version: 6.8.4, in: 5.021s [cds] - [ terminate with ^C ]

In the log files listed below you will find more information from the deployment to CAP. smart-converter-api-srv.log smart-converter-api-db-deployer.log OPERATION.log

[jmsrpp]

Hi Oliver,

You opened the issue originally in Discovery Center with the following error message:

"Service \"ConverterService\" has no handler for \"aiProxy\"."

This is the CAP action used to interact with the Gen AI components and requires that you complete a few additional steps. Can you confirm you also completed the steps here:

https://github.com/SAP-samples/azure-openai-aicore-cap-api/blob/main/documentation/02-cap-api/03-attach-aicore.md

Also covered in the Discovery Center card, Attach SAP AI Core to CAP API

If so, please send the latest log from the application in BTP cockpit or CLI. The smart-converter-api-srv.log you attached does not contain the stack trace from the error, which would be helpful if we need to investigate further.

Thanks, Jim

[oliver-d148]

Hi Jim,

I did the steps under "02-cap-api/03-attach-aicore.md".

As I wanted to test this application with cds watch, it didn't worked and I got the error message "Service "ConverterService" has no handler for "aiProxy"." Also I had not the error that the api-key is missing after opening the local service in the browser.

Then I added the super.init(); in the init method in the converter.ts file and changed "cds watch" into "cds-ts watch --profile hybrid" everything worked. Including the POST request to http://localhost:4004/converter/aiProxy with an inference to the ai proxy and the text from chatgpt.

But after deploying it to the CAP cf, it didn't worked and I got the same error message, that I couldn't find the "selfmade" functions. It can just call the entities of the db.

This is the POST request to the service:

deployment aiProxy query

POST .......ondemand.com/converter/aiProxy content-type: application/json api-key: ...

{ "prompt": "What is SAP known for" }

and my response:

HTTP/1.1 501 Not Implemented content-length: 125 content-type: application/json;odata.metadata=minimal date: Tue, 07 May 2024 16:03:05 GMT odata-version: 4.0 x-correlation-id: b5d10653-3c9a-4613-4641-0c55af9222df x-powered-by: Express x-vcap-request-id: b5d10653-3c9a-4613-4641-0c55af9222df strict-transport-security: max-age=31536000; includeSubDomains; preload; connection: close

{ "error": { "code": "501", "message": "Service \"ConverterService\" has no handler for \"aiProxy\".", "@Common.numericSeverity": 4 } }

and this is the smart-converter-api-srv.log after the POST request.

smart-converter-api-srv (1).log

Thanks, Oliver

[jmsrpp]

Thanks Oliver. Strange that this error doesn't show up in the srv log ... not sure what we're missing. Also, await super.init() is already part of the source:

https://github.com/SAP-samples/smart-co2nverter-ai/blob/main/src/api/srv/converter.ts#L16

Did you add it again or move it to a different place? The only error in the log is when you tried to call the entity Challenges:

[cds] - Error: No database connection at async _readCollection ...

I'll also tag my colleague @julian-schambeck in case he has any inputs here as well. In the meantime, can you check the application bindings in CF to make sure all services are bound correctly?

[oliver-d148]

Thanks for the advise. Maybe I missed it or deleted the line of code. The init method should be fine now.

Everything has started in CF and the services are bound correctly.

Could it be a problem that I'm using the trial version of the BTP?

[julian-schambeck]

@oliver-d148 Hi Oliver, note that you need an AI Core instance with extended plan to leverage any of its generative AI features. So the trial version of BTP won't be sufficient anymore.

Recently, we slightly changed the configuration steps needed to connect to AI Core. The callLargeLanguageModelDestination method inside the service implementation (converter.ts) is now responsible for establishing the connection to AI Core - as opposed to the aiProxy action used previously for that.

We will remove the latter soon to not cause any further confusion.

[oliver-d148]

Hi Julian,

the connection from my BTP trial with cds-ts watch over the configurred destination works. Therefore i can call the AI Launchpad / Core. But it don't work with the deployment.

I have noticed that you have changed the steps so that it works with the Generative AI Hub and I rechanged these steps so it still works with the "old version".

Sorry for the missunderstanding.

[julian-schambeck]

@oliver-d148 What endpoint/functionality of AI Core do you expect to use then currently - if not for the generative AI features: in other words, what does your aiProxy handler implementation currently look like?

Because in the sample it is trying to call a chat completion endpoint (so generative AI), and as said, the aiProxy shouldn't really work right now, not locally nor on the deployed version.

[oliver-d148]

@julian-schambeck My aiProxy handler should be the same as the mission was before your change to the Generative AI hub.

Even if it is the same, i would suggest there would occur the same error message "Service "ConverterService" has no handler for "aiProxy"." as I got it now.

This is a sample of the implementation of the converter.cds the converter.ts file:

....

/**

• Action forwarding prompt to Azure OpenAI services through SAP AI Core provided proxy
• @param {Request} req

• @returns GPTTextResponse { text : string } */ private aiProxy = async (req: Request): Promise<{ text: string } | undefined> => { const { prompt } = req.data; const response = await this.callAIProxy(prompt); return { text: response["choices"][0].message.content }; };

  /**

• Forwards prompt of the payload via a destination (mapped as AICoreAzureOpenAIDestination) through an SAP AI Core deployed service to Azure OpenAI services
• @param {string} prompt
• @returns raw response from Azure OpenAI services for Completions (see https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#example-response-2) */ private callAIProxy = async (prompt: string): Promise<any | undefined> => { const openai = await cds.connect.to('AICoreAzureOpenAIDestination'); const payload = { ...GPT_PARAMS, messages: [ { role: "system", content: "Assistant is a large language model trained by OpenAI" }, { role: "user", content: prompt } ] }; // @ts-ignore const response = await openai.send({ // @ts-ignore query: "POST /v2/chat-completion", data: payload }); return response; };

....

The problem is, that the service, these function and the server.ts file, itself does not work properly. My first thought was, that i made a mistake in some of these steps. But the local test with "cds-ts watch --profile hybrid" works. After calling the local service with "http://localhost:4004" i got the error message that the api-key is missing. I tried it with the normal POST request and got the answer from chat-gpt. Therefore I knew, it worked locally.

Maybe I'm missing something fundamentally with the local cds and the deployment from applications.

[oliver-d148]

Hey,

do you have some news for me?

Best regards, Oliver

[julian-schambeck]

@oliver-d148 Hi Oliver, which Node version (see node --version) are you using? Also, does the error "Service "ConverterService" has no handler for "aiProxy"" happen only for the aiProxy handler - or also for other custom functions/actions implemented in converter.ts?