anykeyh / clear

Advanced ORM between postgreSQL and Crystal
https://github.com/anykeyh/clear
MIT License
272 stars 34 forks source link

unable to save changed record #211

Closed sa-0001 closed 3 years ago

sa-0001 commented 3 years ago

could anyone help shed some light, on why the following code does not save any changes?

::Log.builder.bind "clear.*", Log::Severity::Debug, Log::IOBackend.new

class BookModel
    include Clear::Model
    @@schema = "bk"
    @@table = "books"

    column id : Int32, primary: true

    column image_1 : String?
    column image_2 : String?
    column image_1_previous : String?
    column image_2_previous : String?
end

BookModel.query.where(%[
    deleted = false
    /* ... other filters here */
]).each_with_cursor(batch: 100) do |row|
    pp row.changed? # false

    row.image_1_previous = row.image_1
    row.image_2_previous = row.image_2

    row.image_1 = normalize_image row.image_1_previous
    row.image_2 = normalize_image row.image_2_previous

    pp row.changed? # true
    row.save! # correct UPDATE query is logged
    pp row.changed? # false
end

printing the state of the record, i see that the columns were changed and have the expected values, and row.changed? is true. however, neither row.save nor row.save! actually persists any changes.

could each_with_cursor somehow be creating read-only models, or could it implicitly be within a transaction, ...?

puzzlingly, running the same script on a different server, has the correct result (columns are updated in the database).

UPDATE: after deleting /lib and shard.lock and reinstalling dependencies, it now fails on all servers.

sa-0001 commented 3 years ago

more tests:

  1. if i select the same id again in a new model (new_row = BookModel.find! row.id), the updated values are present
  2. if i make further changes t it and save again, those changes are also not persisted
  3. performing the same operation outside of the cursor, updated the table as expected

this leads me to believe that within the cursor, changes are written to a cached model but not to the database, despite the expected UPDATE query being logged.

sa-0001 commented 3 years ago

mystery solved - normally a cursor (without WITH HOLD) can only be used within a transaction, so for this script a cursor can't be used, because there are millions of rows that might take days to process, all in the same transaction.

anykeyh commented 3 years ago

Sorry for the late update. Indeed, you should use pagination API in this case. This might however cause problems if other processes are creating/updating the records at the same time.