search

etianen / django-watson

Full-text multi-table search application for Django. Easy to install and use, with good performance.
BSD 3-Clause "New" or "Revised" License
1.2k stars 129 forks source link

Using non numeric primary keys #297

Open girardinsamuel opened 2 years ago

girardinsamuel commented 2 years ago

Hi !

I am using Django Hash ID for my primary keys of my models (https://github.com/nshafer/django-hashid-field). When building indexes of the models django-watson is expecting numeric primary keys. Would it be possible to handle both string and numeric primary keys ? Or do you advise a better strategy for using django-watson in my case ?

Here is the error log when running buildwatson command:

Traceback (most recent call last):
  File "/Users/johndoe/repos/myproject/application/./manage.py", line 22, in <module>
    main()
  File "/Users/johndoe/repos/myproject/application/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 187, in handle
    refreshed_model_count += rebuild_index_for_model(
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 74, in rebuild_index_for_model
    _bulk_save_search_entries(iter_search_entries(), batch_size=batch_size_)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 207, in _bulk_save_search_entries
    search_entry_batch = list(islice(search_entries, 0, batch_size))
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 47, in iter_search_entries
    for search_entry in search_engine_._update_obj_index_iter(obj):
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 510, in _update_obj_index_iter
    object_id_int, search_entries = self._get_entries_for_obj(obj)
  File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 480, in _get_entries_for_obj
    object_id_int = int(obj.pk)

ValueError: invalid literal for int() with base 10: 'OzZYA4y3q9X5VMGg'

Here OzZYA4y3q9X5VMGg is the hash id of my model (which is the primary key).

Thank you very much 🙏 !

etianen commented 2 years ago

django-watson supports non-integer primary keys.

Unfortunately, your hashid field subclasses IntegerField (see https://github.com/nshafer/django-hashid-field/blob/master/hashid_field/field.py#L188), so django-watson is assuming it will have an integer type. Which is a pretty fair assumption, IMO.

Which django-hashid-field is implemented as an IntegerField that does not accept integers, this is going to be a problem for django-watson.

On Wed, 31 Aug 2022 at 10:38, Samuel Girardin @.***> wrote:

Hi !

I am using Django Hash ID for my primary keys of my models ( https://github.com/nshafer/django-hashid-field). When building indexes of the models django-watson is expecting numeric primary keys. Would it be possible to handle both string and numeric primary keys ? Or do you advise a better strategy for using django-watson in my case ?

Here is the error log when running buildwatson command:

Traceback (most recent call last):

File "/Users/johndoe/repos/myproject/application/./manage.py", line 22, in 

main()

File "/Users/johndoe/repos/myproject/application/./manage.py", line 18, in main

execute_from_command_line(sys.argv)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/init.py", line 446, in execute_from_command_line

utility.execute()

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/init.py", line 440, in execute

self.fetch_command(subcommand).run_from_argv(self.argv)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv

self.execute(*args, **cmd_options)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute

output = self.handle(*args, **options)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 187, in handle

refreshed_model_count += rebuild_index_for_model(

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 74, in rebuild_index_for_model

_bulk_save_search_entries(iter_search_entries(), batch_size=batch_size_)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 207, in _bulk_save_search_entries

search_entry_batch = list(islice(search_entries, 0, batch_size))

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/management/commands/buildwatson.py", line 47, in iter_search_entries

for search_entry in search_engine_._update_obj_index_iter(obj):

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 510, in _update_obj_index_iter

object_id_int, search_entries = self._get_entries_for_obj(obj)

File "/Users/johndoe/.pyenv/versions/myproject_application/lib/python3.9/site-packages/watson/search.py", line 480, in _get_entries_for_obj

object_id_int = int(obj.pk)

ValueError: invalid literal for int() with base 10: 'OzZYA4y3q9X5VMGg'

Here OzZYA4y3q9X5VMGg is the hash id of my model (which is the primary key).

Thank you very much 🙏 !

— Reply to this email directly, view it on GitHub https://github.com/etianen/django-watson/issues/297, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEKCHN3PK26BFLBTNS4VDV34R3BANCNFSM6AAAAAAQBGKX5I . You are receiving this because you are subscribed to this thread.Message ID: @.***>