Custom Skill Entity Recognition with Azure AI Studio

amitkalay commented 1 month ago

Purpose

This PR adds a custom skill to do entity recognition from a piece of text using the Phi LLM deployed in Azure AI Studio.

Does this introduce a breaking change?

[ ] Yes
[x ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Information provided in the README, but generally send an HTTP request using the file included in this PR.

Test Plan

I sent a POST request to the /api/entity_recognition endpoint and got the following output:

HTTP/1.1 200 OK
Date: Thu, 07 Oct 2024 20:29:30 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: close
Cache-Control: no-cache,no-store
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Set-Cookie: .Tunnels.Relay.WebForwarding.Cookies=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; secure; samesite=none,.Tunnels.Relay.WebForwarding.Cookies=CfDJ8E0FHi1JCVNKrny-ARCYWxM-D2lCVBfcNJJ-AoLveoZR0EYildsrrCG7Cdy531e5soHN8de1cxCc4of1aVs8PbsPdvs0NVR7HXoHsEd3ZG4Pb2NdDa82NFh13mgjij4ZD6vgg-3LgFA9mt8vyfecccrcTfur9yABsFpC5hHvJGwWE9XpytKXe__-WkLI2B8ykPv3nn5k8v1tAqP_pQ81sgG_oMsOfnk-KNOYwmE1gy0yZ06qgav4B5GBETwBCrujlw7M5hddCgTnZIDUC9rXVto1L-pSONpYXJlahd5slX5FKHI7MHrHP279IVaoU1U2EP7NXQlPg_Ef3s0C4mN274AMOKoqObaXi_Bh80ZaDFAdmlGmDbq9iz6Qc29r45F_8VN8R2v7mJcdLAPUdwBNCFIomF4OvBJ1uURYmgkOgdlkBjX5fJGC8PddoXEzFscBukwuDv4OjtLaw6N96bHoREZqWpeUfkgvcCCvg9rmPUAa0ga6jrFaVYYV6OXDGTyEzudDqJptrfmJdW6oocUaBAic5OmuYGIrF_vu4OyqlAFWp0CtO-zttIBVN-vEgEGhxlKrk338T4YdjfMbnSx_Q6jgEs5hGNCE4GSSYUJG4ChDYs_RZUB66NZcmS_NCmNr81sMqvl2ZB1thL6U2m7AjxDWRmhITfOtL9yYJPQ-xP4ZzJiYC_EA7Sf3YAc3vXWGLtqBRMSx8iTHp6rzQYTDy7qllDMreDqB87w82x2t5_2s22fjAKH_SgD6_lqpRj2FWgTqBUl8nHRQLeJ0Nka_tOhYID-favouLHUdudgI7svjlASu9pNlFBNHGIIeqQ-irKalqahAxQ7cLtS0-CmC02kS_GGd_ZGSCofiV1GBf7SYvONW6emsAbWZxoIQi__9Aav9s75gQ_s6bV_-KO0iodSoWjzLEJw3XNvvuR1QYPTCLUeW17_XucQQ-j7Hk5E4qfKN5V7EEV0U-KW8P_10nf4JozTF_GvT63DyNHCAO7n9e6qObnC2ZSDtD7YR-7BCfg; path=/; secure; samesite=none
X-Content-Type-Options: nosniff
RateLimit-Limit: HttpRequestRatePerPort:1500/m
RateLimit-Remaining: HttpRequestRatePerPort:1499
RateLimit-Reset: HttpRequestRatePerPort:45s
X-Report-Abuse: https://msrc.microsoft.com/report/abuse
x-ms-ratelimit-limit: 1500
x-ms-ratelimit-remaining: 1499
x-ms-ratelimit-used: 1
x-ms-ratelimit-reset: 
X-Robots-Tag: noindex, nofollow
Referrer-Policy: same-origin
VsSaaS-Request-Id: d80210cc-3748-436c-b191-59cbfd5c5201
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Served-By: tunnels-prod-rel-usw3-v3-cluster

{
  "values": [
    {
      "warnings": null,
      "errors": [],
      "recordId": "0",
      "data": {
        "entities": " {\n  \"people\": [\"Dario Amodei\"],\n  \"addresses\": [],\n  \"email_addresses\": [],\n  \"engineering_job_titles\": [\"AI models\", \"reasoning models\"],\n  \"funding_rounds\": [\n    {\n      \"company\": \"OpenAI\",\n      \"amount\": \"$6.6 billion\",\n      \"valuation\": \"$157 billion\"\n    },\n    {\n      \"company\": \"xAI\",\n      \"amount\": \"$6 billion\",\n      \"valuation\": \"Not specified\"\n    }\n  ],\n  \"investors\": [\n    {\n      \"name\": \"Thrive Capital\",\n      \"amount\": \"$1 billion\",\n      \"special_deal\": \"Invest another $1 billion next year at the same valuation if the AI firm hits a revenue goal\"\n    }\n  ],\n  \"company_structures\": [\n    {\n      \"company\": \"OpenAI\",\n      \"structure\": \"for-profit\",\n      \"capital_gains_limit\": \"100x\",\n      \"restructure_deadline\": \"2 years\"\n    },\n    {\n      \"company\": \"Anthropic\",\n      \"structure\": \"public benefit corporation\",\n      \"consideration\": \"becoming a public benefit corporation\"\n    }\n  ],\n  \"restrictions\": [\n    {\n      \"company\": \"OpenAI\",\n      \"restriction\": \"investors asked to avoid backing rival start-ups such as Anthropic and Elon Musk's xAI\"\n    }\n  ]\n}"
      }
    }
  ]
}

arv100kri commented 1 month ago

Hey a meta comment about the organization of the work...

I think we shouldn't break things down this way. Instead of entityskill, imageskill and summary skill - I think we should have

AOAIInferenceSkill and CustomInferenceSkill

And each skill, should do all of the 3 things - with the backing models and client calls varying depending on if it's AOAI or the "Other" models.

amitkalay commented 1 month ago

@arv100kri - I understand the general organization of the work/code you had in mind for this a lot better now. Since this code still does something valuable for us (entity recognition), I will follow up in a future PR to refactor the codebase to do things to align with what was envisioned.

Azure-Samples / azure-search-power-skills