search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.15k stars 390 forks source link

Is there a `.railsignore`? #277

Closed Wildshire closed 2 weeks ago

Wildshire commented 9 months ago

Hello

Very quick question / topic. Is there a .nemoignore file to be added to the projects when loading LLMRails for example? Thing is, if I am doing some testing with colang files or a kb, sometimes I want to not load those files specifically. Only way I found to solve the issue is to delete files or folders. Perhaps there is another simpler workaround that I have not noticed yet.

Thanks in advance

Wildshire commented 8 months ago

Hello

I saw that in the last update to 0.8.0 you added these configurations. This is a good step up into this issue.

Keep up the good work :)

PD: Leaving conversation open just in case something else is planned for the future

drazvan commented 8 months ago

Hi @Wildshire! Yes, that can be used. Also, #356 adds support for symlinks, so that could potentially be used as well. If you have a bit of time to contribute the support for a .nemoignore, I can guide you. Should not be a difficult one.

Wildshire commented 8 months ago

Hi @drazvan. Sorry for the late response. I might be able to help with this one. Where should I look into?

drazvan commented 7 months ago

First, let's call it .railsingore. I think the two main places where we want to apply the filtering are:

You should create a helper function that we can invoke from various places.

Thanks!

ajanitshimanga commented 4 months ago

Hi @drazvan! Yeah, I'd like to look into this issue as a first contribution! Some guidance would be greatly appreciated as I learn the codebase. Looking forward to tackling this!

drazvan commented 4 months ago

@ajanitshimanga : the discussion above should provide enough details. Let me know if you need additional guidance.

ajanitshimanga commented 2 months ago

Looking into this now 👍

ajanitshimanga commented 1 month ago

Hey @drazvan ,

Is it preferable for the filtering of .co files listed in .railsignore to take place on invocation of _load_path or in _join_config for llm/config.py?

Above you referenced _join_config in the first link and it seems to also be nicely handled near:

https://github.com/NVIDIA/NeMo-Guardrails/blob/5cbb2b63a509697b99adab9eaa4b9aa7c0a9c543/nemoguardrails/rails/llm/config.py#L583

where we can filter .co files that are found in .railsignore explicitly during loading rather than hidden in config joining? I have started implementation as you described above in _join_config but wanted to ask on this.

Secondly, from your message above it seems that you would like the ActionDispatcher's _load_actions_from_module to filter out .py modules based on the .railsignore as well. Is this correct? So essentially, end users will specify .py files and/or .co files in .railsignore to be filtered for RailsConfig or ActionDispatcher?

Thanks!

Pouyanpi commented 1 month ago

Hi @ajanitshimanga,

Thanks for looking into this. Your approach looks good! 👍🏻

Having support for .py files would also be a nice addition. It would be great if you could add support for some glob patterns, similar to how .gitignore works, to filter out .co or .py files based on the pattern. If your design could be flexible enough to allow support for additional file types in the future, that would be ideal.

I suggest opening a PR so we can review it together and collaborate further if needed.

ajanitshimanga commented 1 month ago

Thanks @Pouyanpi ! I’ll proceed with the information you’ve clarified. I’ll open a PR soon after.

ajanitshimanga commented 1 month ago

Hi @Pouyanpi , I submitted a PR for the changes I've made. I would like to engage on what next steps would look like.

Thanks!