sturdy-dev / semantic-code-search

Search your codebase with natural language • CLI • No data leaves your computer
GNU Affero General Public License v3.0
305 stars 27 forks source link

support utf8 and other encoding type #25

Open bankThanabat opened 1 year ago

bankThanabat commented 1 year ago

fix #24 This pull request adds support for an optional encoding type argument in the embed.py module. The previous version of the module did not properly handle UTF-8 encoded files when used with the do_embed() function. This issue was caused by the parser.parse() method in the _get_repo_functions() function

To resolve this issue, i added support for an optional encoding type argument using the -en or --encoding argument. Users can now specify an encoding type when running the script, and the script will use the specified encoding type when processing files. If no encoding type is specified, the default encoding type 'utf-8' will be used.

The following changes were made to the embed.py module:

These changes should resolve the issue with UTF-8 encoded files and make the script more flexible