Added auxiliary files that can use Zendesk API

[YonghaoZhao722]

Add code that can obtain web page information by using the zendesk API instead of using the default extract Text From function and sitemap.xml. The information is stored in the same format as the “pages”. It should be a list of dictionaries, where the k,v pair should be: {“text”: text_of_support_article, “source”: url_of_the_article}

[mpaepper]

Hi and thanks for the contribution.

Please make sure that the URL that can be used with it is a configurable parameter (e.g. check the ArgumentParser in https://github.com/mpaepper/content-chatbot/blob/main/create_embeddings.py).

Also for the README please don't replace the original README. It's fine to add a section about the new feature, though.

Thank you!

[YonghaoZhao722]

Hi and thanks for the contribution.

Please make sure that the URL that can be used with it is a configurable parameter (e.g. check the ArgumentParser in https://github.com/mpaepper/content-chatbot/blob/main/create_embeddings.py).

Also for the README please don't replace the original README. It's fine to add a section about the new feature, though.

Thank you!

Thanks for your feedback and guidance. I've made the necessary adjustments according to your suggestions:

The URL is now a configurable parameter in the code. I followed the example in the additional README section.

Regarding the README, I've refrained from replacing the original content and instead added a dedicated section to highlight the new feature while preserving the existing information.

[mpaepper]

Great work, thank you @zyh040521

I was just thinking that a lot of the code is actually duplicated, so maybe it would be a good idea to merge the two create_embeddings files and add a configurable option whether to parse using Zendesk.

What do you think?

[YonghaoZhao722]

Cool. I've implemented the changes by merging the code files. Now, by default, it runs in sitemap mode and switches to Zendesk mode only when the -m parameter is specified.

[mpaepper]

@zyh040521 Thank you again for giving your changes back to the community!

It's merged to main now.