Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.63k stars 3.77k forks source link

I'm sorry, but I don't have access to that information - Login and ACL #1509

Open michbalazs opened 3 months ago

michbalazs commented 3 months ago

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ X] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Set up and tested the app using GPT 3.5 and GPT4V. Uploaded a lot of docs using the prepdocs.py, which could be retrieved by web app. Then added in authentication and ACL - forcing users to login. ran the manage acls scripts. looked up a document to check my access and now I just get I dont know as a response or I'm sorry, but I don't have access to that information.. I can see from the thought process that the search is retrieving an empty set. Do i need to reindex something ?

Any log messages given by the failure

Expected/desired behavior

It returned an answer based on the documents searched as before.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here. azd version 1.7.0 (commit 49d6adc2efb178083f61822e6b4715258560803d)

Versions

Mention any other details that might be useful


Thanks! We'll be in touch soon.

michbalazs commented 3 months ago

I should say it works fine in openai studio

pamelafox commented 3 months ago

Hm, I think the next thing I would try is:

Perhaps the IDs aren't quite set correctly?

cc @mattgotteiner

michbalazs commented 3 months ago

Thanks. It seems okish

  1. I have access

  2. Querying the search index for the file { "@odata.context": "https://gptkb-xxxxx.search.windows.net/indexes('gptkbindex')/$metadata#docs(*)", "@search.nextPageParameters": { "search": "had course guide.pdf", "select": "sourcefile,oids,groups", "filter": "sourcefile eq 'had course guide.pdf'", "skip": 50 }, "value": [ { "@search.score": 8.523854, "sourcefile": "had course guide.pdf", "oids": [], "groups": [] }, { "@search.score": 7.436149, "sourcefile": "had course guide.pdf", "oids": [], "groups": [] }, (Multiple similar rows with lower scores ?)

  3. Query was What courses does Hadlow college offer in animal management { "search": "What courses does Hadlow college offer in animal management", "queryType": "semantic", "semanticConfiguration": "default", "captions": "extractive", "answers": "extractive|count-3", "queryLanguage": "en-US" }

{ "@odata.context": "https://gptkb-hdtqjlhd3jjw4.search.windows.net/indexes('gptkbindex')/$metadata#docs(*)", "@search.answers": [ { "key": "file-had_course_guide_pdf-68616420636F757273652067756964652E706466-page-42", "text": "full-time courses offer at hadlow campus course titlelevelfdsc in applied animal behavioural science & welfare4 & 5fdsc in zoo husbandry & animal management4 & 5fdsc in equine training and management4 & 5fdsc in horticulture - commercial4 & 5bsc top up in equine training and management6bsc top up in applied animal …", "highlights": "full-time courses offer at hadlow campus course titlelevelfdsc in applied animal behavioural science & welfare4 & 5fdsc in zoo husbandry & animal management4 & 5fdsc in equine training and management4 & 5fdsc in horticulture - commercial4 & 5bsc top up in equine training and management6bsc top up in applied animal …", "score": 0.83056640625 }, { "key": "file-had_course_guide_pdf-68616420636F757273652067756964652E706466-page-63", "text": "The College courses include: Full-time Further Education College delivered course - Levels 1-3 in Agriculture, Agricultural Engineering, Animal Management, Equestrian Studies, Fisheries Management, Floristry and Horticulture.", "highlights": "The College courses include: Full-time Further Education College delivered course - Levels 1-3 in Agriculture, Agricultural Engineering, Animal Management, Equestrian Studies, Fisheries Management, Floristry and Horticulture.", "score": 0.81494140625 },