M1ha-Shvn / django-pg-bulk-update

Django manager for performing bulk update operations in PostgreSQL database
BSD 3-Clause "New" or "Revised" License
39 stars 13 forks source link

Update or create with key_fields doesn't allow use of returning='*' #89

Closed Toika closed 2 years ago

Toika commented 2 years ago

Call:

created_cp = ConversationPart.objects.using(app.shard).pg_bulk_update_or_create(parts_to_create,
                                                                                key_fields=('external_id', 'conversation_id'),
                                                                                returning='*'
                                                                                )

Key fields do form a unique index for the table and replacing '*' with a tuple of field names do helps.

Here's a traceback:

Traceback (most recent call last):
  File "/home/vagrant/venv/lib/python3.8/site-packages/django/db/models/options.py", line 608, in get_field
    return self.fields_map[field_name]
KeyError: '*'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "import_data.py", line 733, in <module>
    import_conversation_parts_batch(app_id, conversation_parts_batch)
  File "import_data.py", line 589, in import_conversation_parts_batch
    created_cp = ConversationPart.objects.using(app.shard).pg_bulk_update_or_create(parts_to_create,
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_clickhouse/models.py", line 88, in pg_bulk_update_or_create
    return self._decorate_method('pg_bulk_update_or_create', 'update', args, kwargs)
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_clickhouse/models.py", line 80, in _decorate_method
    result = func(*args, **kwargs)
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/manager.py", line 151, in pg_bulk_update_or_create
    return bulk_update_or_create(self.model, values, key_fields=key_fields, using=using,
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/query.py", line 981, in bulk_update_or_create
    batched_result = batched_operation(batch_func, values,
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/utils.py", line 174, in batched_operation
    r = handler(*args, **kwargs)
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/query.py", line 906, in _insert_on_conflict_no_validation
    ret_sql, ret_params = _returning_query_part(model, ret_fds)
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/query.py", line 499, in _returning_query_part
    return "RETURNING %s" % ', '.join('"%s"' % fd.get_field(model).column for fd in ret_fds), []
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/query.py", line 499, in <genexpr>
    return "RETURNING %s" % ', '.join('"%s"' % fd.get_field(model).column for fd in ret_fds), []
  File "/home/vagrant/venv/lib/python3.8/site-packages/django_pg_bulk_update/types.py", line 43, in get_field
    return model._meta.get_field(self.name)
  File "/home/vagrant/venv/lib/python3.8/site-packages/django/db/models/options.py", line 610, in get_field
    raise FieldDoesNotExist("%s has no field named '%s'" % (self.object_name, field_name))
django.core.exceptions.FieldDoesNotExist: ConversationPart has no field named '*'
M1ha-Shvn commented 2 years ago

This is not a problem of this library. It is a problem of django-clickhouse _decorate_method(...) function, replacing '*' with ['id', '*']