feat: added script to pull documents for RAG

[mfaizanse]

Description

Changes proposed in this pull request:

• added script to pull documents for RAG

Related issue(s)

• https://github.com/kyma-project/kyma-companion/issues/199

[github-actions[bot]]

Note(s) for PR Auther:

• The integration test will be skipped for the PR. You can trigger it manually after adding the label: run-integration-test.
• The evaluation test will be skipped for the PR. You can trigger it manually after adding the label: evaluation requested.
• If any changes are made to the evaluation tests data, make sure that the integration tests are working as expected.
• If any changes are made to how to run the unit tests, make sure to update the steps for unit-tests in the create-release.yml workflow as well.

  Note(s) for PR Reviewer(s):

• Make sure that the integration and evaluation tests are working as expected.

[muralov]

I have found the following files after the docs are fetched, which can be mostly excluded:

api-gateway/docs/release-notes it has mostly release notes.

api-gateway/docs/contributor/adr/adr-template.md/docs/adr-template.md:

# <!--- Provide title -->

## Status
<!--- Specify the current state of the ADR, such as whether it is proposed, accepted, rejected, deprecated, superseded, etc. -->

## Context
<!--- Describe the issue or problem that is motivating this decision or change. -->

## Decision
<!--- Explain the proposed change or action and the reason behind it. -->

## Consequences
<!--- Discuss the impact of this change, including what becomes easier or more complicated as a result. -->

Lots of README.md had almost little meaningful contents. For example: docker-registry/docs/user/resources/README.md:

# Resources

In this section, you can find the custom resources (CR) used in the Docker Registry module.

Maybe a follow-up PR to clean-up these files?

[mfaizanse]

I have found the following files after the docs are fetched, which can be mostly excluded: api-gateway/docs/release-notes it has mostly release notes. api-gateway/docs/contributor/adr/adr-template.md_garbage/docsadr-template.md:

# <!--- Provide title -->

## Status
<!--- Specify the current state of the ADR, such as whether it is proposed, accepted, rejected, deprecated, superseded, etc. -->

## Context
<!--- Describe the issue or problem that is motivating this decision or change. -->

## Decision
<!--- Explain the proposed change or action and the reason behind it. -->

## Consequences
<!--- Discuss the impact of this change, including what becomes easier or more complicated as a result. -->

Lots of README.md had almost little meaningful contents. For example: docker-registry/docs/user/resources/README.md_garbage:

# Resources

In this section, you can find the custom resources (CR) used in the Docker Registry module.

Maybe a follow-up PR to clean-up these files?

I have excluded some files. But for automation of this, we should have some check before we create embeddings for a chunk.