CGSmith-LLC / shipwise-api

Shipwise API
Other
0 stars 0 forks source link

Order attributes #170

Open bohdan-vorona opened 1 year ago

bohdan-vorona commented 1 year ago

https://github.com/CGSmith-LLC/shipwise-api/issues/165

Listing:

attrs-listing

View:

attrs-view

Add/Update:

attrs-add-update

API:

attrs-api

bohdan-vorona commented 1 year ago

@samdark

I needed the method executeAfterFind since I have the field order_attributes_array I use for quick accessing the attributes in array format like this:

return implode(', ', array_map(function ($attr) {
                                return Html::a(Html::encode($attr), Url::to(['index', 'OrderSearch[order_attributes]' => Html::encode($attr)]));
                            }, $model->order_attributes_array));

But anyway I moved it to the JsonAttributeBehavior and changed the name.

cgsmith commented 1 year ago

Working on checking this but my local machine is running slow. Trying to reduce the dataset.

samdark commented 1 year ago

Won't the real dataset be even larger?

cgsmith commented 1 year ago

Yes. And I'm not sure why it is running slow. I'm going to copy production to another database on Digital Oceans server and run the migration against it.

bohdan-vorona commented 1 year ago

@cgsmith

Working on checking this but my local machine is running slow. Trying to reduce the dataset.

After adding the field?

cgsmith commented 1 year ago

Yes. And I'm not sure why it is running slow. I'm going to copy production to another database on Digital Oceans server and run the migration against it.

It just sits there while trying to add the field. I tried from the mysql cli too.

samdark commented 1 year ago

A good idea is to give @bohdan-vorona a DB dump so he reproduce/fix it.

cgsmith commented 1 year ago

Good idea. I'll get a sql dump today for you @bohdan-vorona

bohdan-vorona commented 1 year ago

@cgsmith @samdark It turned out a pretty tricky thing...

On my local machine it took almost 300 seconds (5 minutes) to execute the query:

ALTER TABLE orders ADD order_attributes JSON NULL DEFAULT NULL AFTER notes;


To be short, it will take lots of time to add a new field to the table orders due to lots of records. I tried all the variants I've found on the internet like setting a different mode, locking/unlocking, creating a tmp table and then moving data between tables, etc. I even tried to remove FULLTEXT due to restrictions for one of the modes presented here - https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html

The "faster" way: (except the direct query execution)

  1. Remove FULLTEXT.
  2. Add the field using one of the existing algorithms (the link above). Time: ~4 mins. Only 1 minute difference.
  3. Return back FULLTEXT. Time: ~4.5 mins.

Total time approximately: ~9-10 minutes (on my local machine).


So:

Option A. Just apply the migration as is. Probably for this purpose, it would be better to have a column, not a separate table. In addition, as a separate task, I'd propose to:

Option B. Create a new table - order_attribute(s).

Option C. Create a new table but with keeping in mind then potentially we can extend the table with other columns in future. Let's name it order_extra or order_param(s) with the first field attributes needed for this task.

BTW it takes lots of time also removing the column 😁

samdark commented 1 year ago

That sounds alright. @cgsmith let's apply that during "maintenance window"?