statamic / eloquent-driver

Provides support for storing your Statamic data in a database, rather than flat files.
https://statamic.dev/tips/storing-content-in-a-database
MIT License
104 stars 73 forks source link

Update entry order when saving a tree and collections are stored in files #158

Closed jasonvarga closed 1 year ago

jasonvarga commented 1 year ago

Fixes #139

This adds an event listener that does the exact same thing as the core's UpdateStructuredEntryOrder listener.

It'll update the order column of entries when you save a tree.

It will only do it if entries are configured to be in Eloquent, and Collections are configured to not be.

modrictin commented 1 year ago

Hey @jasonvarga ,

what happened before you introduced this change?

I'm thinking of removing it for my use case. I tried to update to latest version and I have some issues with this update.

More on the issue here, in my discord message:

_Hey everyone,

I'm currently facing an issue while upgrading from Eloquent driver v1.2 to v2.3.1 and could really use some assistance.

The problem stems from the introduction of an order column in the database. Whenever I save an entry, the UpdateStructuredEntryOrder listener gets triggered because I'm exclusively using the Eloquent driver for managing the entries. As a result, the Eloquent CollectionRepository tries to update the order column for all entries in that collection.

Now, here's the challenge—I have a large number of entries, with over 110,000 in my database, and a few thousand in a single collection. Unfortunately, using the queue driver sync to save the entry isn't a viable solution as it tries to process an enormous amount of entries, causing significant delays. While I'm using a different queue driver in my production environment, I still want to avoid this situation altogether.

I'm looking for any advice, ideas, or insights on how to handle this issue effectively. One thought I had was to remove the logic responsible for updating the order column for the saved entries.

Before proceeding, I have a few questions:

Is it true that the order column is exclusively used for managing the order of entries within a collection in the EntriesController::index() method? Are there any other components or processes in my system that rely on the order column?

If I remove the logic that updates the order column, will it impact any other functionality or disrupt the expected behavior of my application?

Thank you in advance for your help! How did this functionality even work before introducing this order column. If it did not, that is ok with me, I cold just remove it then

Thank you so much! Tin_

ryanmitchell commented 1 year ago

The order is only used in a manually ordered collection - I suspect if you have that many entries you’re not ordering it manually?

the event should not run if you turn off the sortable config setting for the collection.

modrictin commented 1 year ago

Hey @ryanmitchell that's what I thought!

Thank you so much for the confirmation 😊

ryanmitchell commented 1 year ago

If you are finding it’s running on a not structured collection that’s definitely a big and we should look into that.

modrictin commented 1 year ago

Hey @ryanmitchell ,

I did not find anything unordinary, but I'm still experiencing slow entry saving.

Apparently this is causing it: image

Why is this introduced all of a sudden?

I can see that we are now saving all of the data to localized entries as well.

I have 44 locales unfortunately, and each save takes a while :(

Do you have any ideas on how to approach this?

Thanks, Tin

ryanmitchell commented 1 year ago

It was a point of difference between stache and eloquent and needed to be addressed to ensure compatability.

You could always bind your own entryrepository and remove the bits you don’t want.