search

chatdoc-com / chatdoc-com.github.io

https://api-reference.chatdoc.com
2 stars 0 forks source link

Chatdoc API markdown issue #4

Open spseon opened 2 weeks ago

spseon commented 2 weeks ago

I am currently using Chatdoc API to read multiple markdown documents.

When I upload multiple markdown documents to regular Chatdoc(not API), it returns accurate answers as well as reference pages.

However, when I upload the same markdown documents to ChatdocAPI-generated platform, it's returning inaccurate answers and reference pages. The answers are so inaccurate that it seems as if it is understanding the text.

(I used the same chatgpt version(gpt 4.0) to test on both Chatdoc and Chatdoc API)

We'd like to get similar outputs using Chatdoc API as we would when using regular Chatdoc. What could be the causing factor for such a huge discrepancy? Is there any way we can improve the API output?

spseon commented 2 weeks ago

Please, give us feedback as soon as possible. Our service development has stopped due to this issue and we need this issue to be solved as soon as possible.

spseon commented 2 weeks ago

food-laws-chatbot-POC-main.zip

Here is our code for reference.

cumtyc commented 2 weeks ago

Could you please provide a sample document and corresponding questions?

spseon commented 2 weeks ago

Our documents and questions are in Korean. Would that work as well?

For reference, this is the site we created using your API: https://dsx6dsyv-poc.vercel.app

spseon commented 2 weeks ago

Here are the sample files and questions we used to test on API and Chatdoc. Let us know if files in Korean doesn't work for you.

법령모음_MD-20240927T043931Z-001.zip

spseon commented 2 weeks ago

For reference, here is an example :

Chatdoc

Custom API

The same thing happens with English documents:

Chatdoc

Custom API

Custom API does not always return wrong answers. However, even when it does return relevant answers, it is still different from Chatdoc answers(Chatdoc returns more accurate answers). We are aware that chatGPT can give different answers for the same question, but we don't think that is the problem since each of Chatdoc and custom API consistently returns same answers for the same question. Also, custom API consistently provides too many reference pages, most of which are irrelevent.

Following are the files we uploaded for the test(we uploaded all the files in one collection):

We appreciate your help. Thank you.

cumtyc commented 2 weeks ago

We have updated our API service and switched the default model to gpt-4o-mini. You can try it out to see if it resolves your issue.

spseon commented 1 week ago

Thank you very much for the update. It seems to work much better now!

There is an additional issue regarding detailed citation('detailed_citation): 1) The citation number is displayed at the end of each sentence. However, it is not clickable(it's not linked to the text on the Document Viewer)

image

2) Sometimes, the detailed citation does how show at the end of each sentence whereas it does for Chatdoc.

spseon commented 1 week ago

Hello, after doing some further testing, we noticed some issue with the answers.

When we entered the prompt "Please tell me the reference amount for one serving of jam(잼류의 1회 섭취 참고량을 알려줘)" in Chatdoc, it used to extract information from the table and return correct answer("20g"). It was only the case in custom API that the information could not be found.

However, not the answer cannot be found in both Chatdoc and custom API. Was there any update to Chatdoc? Is there a way we can get correct answers for both? (We tested with this prompt a number of times and Chatdoc returned correct answers everytime.)