NgRestEventBehavior Performance Bug

[mhunesi]

I have a model named Products and I use in frontend. I have a serious performance problem when extend from NgRestModel. When I extend from NgRestModel

When I extend from ActiveRecord

I've reviewed and problem is NgRestEventBehavior. vendor/luyadev/luya-module-admin/src/ngrest/base/NgRestModel.php:79

Profiling

Time	Duration	Category	Info
20:06:18.514	48,0 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.438	38,4 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.029	37,4 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.477	37,2 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.400	36,9 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.550	36,7 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.182	36,6 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.587	36,5 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.220	36,4 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.257	36,4 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.103	36,0 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.364	35,8 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:18.992	35,6 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.067	35,4 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.693	35,3 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.146	35,2 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.625	34,7 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.328	34,6 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.515	33,8 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.729	33,5 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.294	33,2 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products
20:06:19.660	31,9 ms	luya\admin\ngrest\base\NgRestModel::getNgRestConfig	app\modules\catalog\models\Products

Additional infos

Q	A
LUYA Version	2
PHP Version	7.3
Platform	nginx
Operating system	Linux Server

[hbugdoll]

I've reviewed and problem is NgRestEventBehavior. vendor/luyadev/luya-module-admin/src/ngrest/base/NgRestModel.php:79

Hi @mhunesi, you can include your Markdown text nice source code previews of one or more code lines, see https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/creating-a-permanent-link-to-a-code-snippet

The link https://github.com/luyadev/luya-module-admin/blob/a198b86455c510c7ebc37ca843590a2ba52a54eb/src/ngrest/base/NgRestModel.php#L77-L83 results in: https://github.com/luyadev/luya-module-admin/blob/a198b86455c510c7ebc37ca843590a2ba52a54eb/src/ngrest/base/NgRestModel.php#L77-L83

[nadar]

@mhunesi could you please post the full model? Maybe there other things related to that performance leak. How do you foreach those data in Frontend context?

[mhunesi]

Sure. I use ListView Widget. When I comment ngRestScopes method everything's okay.

This Product Model.

<?php

namespace app\modules\catalog\models;

use app\modules\accounting\models\Currency;
use Yii;
use luya\admin\traits\SortableTrait;
use app\modules\catalog\admin\Module;
use luya\admin\ngrest\plugins\SelectModel;
use app\modules\catalog\admin\enums\ProductType;
use app\modules\catalog\admin\enums\ProductStatus;
use luya\admin\ngrest\plugins\CheckboxRelationActiveQuery;
use app\modules\catalog\admin\aws\ProductImageActiveWindow;
use app\modules\catalog\admin\aws\ProductDetailActiveWindow;
use yii\db\Expression;

/**
 * Products.
 *
 * File has been created with `crud/create` command.
 *
 * @property integer $id
 * @property integer $sort
 * @property tinyint $status
 * @property string $sku
 * @property string $type
 * @property integer $unit_id
 * @property tinyint $new
 * @property tinyint $featured
 * @property tinyint $visible_individually
 * @property integer $brand_id
 * @property string $name
 * @property string $ean
 * @property string $upc
 * @property string $jan
 * @property string $isbn
 * @property string $mpn
 * @property string $product_number
 * @property integer $inventdim_group_id
 * @property integer $tax_id
 * @property decimal $width
 * @property decimal $height
 * @property decimal $length
 * @property decimal $weight
 * @property text $images_list
 * @property integer $cover_image_id
 * @property decimal $list_price
 * @property decimal $cost_price
 * @property decimal $sale_price
 * @property int $currency_id
 * @property string $note
 * @property string $short_description
 * @property text $description
 * @property string $slug
 * @property string $meta_title
 * @property string $meta_keywords
 * @property string $meta_description
 * @property integer $created_at
 * @property integer $updated_at
 * @property integer $created_by
 * @property integer $updated_by
 * @property tinyint $deleted
 *
 * @property Brands $brand
 * @property InventDimGroup $inventDimGroup
 * @property Units $unit
 * @property Tax $tax
 * @property Size $sizes
 * @property Color $colors
 * @property ProductImages $images
 * @property ProductVariants[] $variants
 * @property ProductVariants $defaultVariant
 */
class Products extends BaseModel
{
    use SortableTrait;

    /**
     * @inheritdoc
     */
    public $i18n = ['name', 'short_description', 'description', 'meta_title', 'meta_keywords', 'meta_description'];

    /**
     * @var array
     */
    public $categories = [];

    /**
     * @inheritdoc
     */
    public static function tableName()
    {
        return '{{%products}}';
    }

    /**
     * @inheritdoc
     */
    public static function ngRestApiEndpoint()
    {
        return 'api-catalog-products';
    }

    /**
     * @inheritdoc
     */
    public function attributeLabels()
    {
        return [
            'id' => Module::t('ID'),
            'sort' => Module::t('Sort'),
            'status' => Module::t('Status'),
            'new' => Module::t('New'),
            'featured' => Module::t('Featured'),
            'visible_individually' => Module::t('Visible Individually'),
            'brand_id' => Module::t('Brand ID'),
            'sku' => Module::t('SKU'),
            'ean' => Module::t('EAN'),
            'upc' => Module::t('UPC'),
            'isbn' => Module::t('ISBN'),
            'mpn' => Module::t('MPN'),
            'product_number' => Module::t('Product Number'),
            'inventdim_group_id' => Module::t('Invent Dim Group ID'),
            'tax_id' => Module::t('Tax ID'),
            'width' => Module::t('Width'),
            'height' => Module::t('Height'),
            'length' => Module::t('Length'),
            'weight' => Module::t('Weight'),
            'unit_id' => Module::t('Unit ID'),
            'images_list' => Module::t('Images List'),
            'cover_image_id' => Module::t('Cover Image ID'),
            'list_price' => Module::t('List Price'),
            'cost_price' => Module::t('Cost Price'),
            'sale_price' => Module::t('Sale Price'),
            'currency_id' => Module::t('Currency'),
            'note' => Module::t('Note'),
            'short_description' => Module::t('Short Description'),
            'description' => Module::t('Description'),
            'slug' => Module::t('Slug'),
            'meta_title' => Module::t('Meta Title'),
            'meta_keywords' => Module::t('Meta Keywords'),
            'meta_description' => Module::t('Meta Description'),
            'created_at' => Module::t('Created At'),
            'updated_at' => Module::t('Updated At'),
            'created_by' => Module::t('Created By'),
            'updated_by' => Module::t('Updated By'),
            'deleted' => Module::t('Deleted'),
        ];
    }

    /**
     * @return string[]
     */
    public function attributeHints()
    {
        return [
            'weight' => 'gr cinsinden giriniz',
            'height' => 'cm cinsinden giriniz',
            'length' => 'cm cinsinden giriniz',
            'width' => 'cm cinsinden giriniz',
            'sku' => 'Stock Code',
            'upc' => 'Kuzey Amerika’da / GTIN-12',
            'ean' => 'European Article Number',
            'isbn' => 'Universal Product Code',
            'jan' => 'Japonya’da / GTIN-13',
            'mpn' => 'Manufactured Product Code',
            'unit_id' => 'Global UNIT',
        ];
    }

    /**
     * @inheritdoc
     */
    public function rules()
    {
        return [
            [['sort', 'status', 'new','featured','visible_individually','brand_id','inventdim_group_id','tax_id','cover_image_id','created_at','updated_at','created_by','updated_by','deleted','unit_id','currency_id'], 'integer'],
            [['width', 'height', 'length', 'weight', 'list_price', 'cost_price','sale_price'], 'number'],
            [['images_list', 'description'], 'string'],
            [['status', 'new','featured'], 'boolean'],
            [['ean'], 'string', 'max' => 14],
            [['upc'], 'string', 'max' => 13],
            [['jan'], 'string', 'max' => 13],
            [['sku', 'mpn'], 'string', 'max' => 64],
            [['product_number'], 'string', 'max' => 50],
            [['note', 'short_description', 'meta_description'], 'string', 'max' => 1500],
            [['slug', 'meta_title', 'meta_keywords'], 'string', 'max' => 255],
            [['slug','sku'], 'unique'],
            [['product_number','ean'], 'unique'],
            [['slug'], 'filter', 'filter' => 'trim'],
            [
                ['slug'],
                'filter',
                'filter' => function ($value) {
                    $char_map = ['Ş' => 'S',    'İ' => 'I',    'Ç' => 'C',    'Ü' => 'U',    'Ö' => 'O',    'Ğ' => 'G',    'ş' => 's',    'ı' => 'i',    'ç' => 'c',    'ü' => 'u',    'ö' => 'o',    'ğ' => 'g'];
                    return str_replace(array_keys($char_map), $char_map, mb_strtolower($value, 'UTF-8'));
                }
            ],
            [['currency_id','inventdim_group_id','categories','unit_id','tax_id','status','name','brand_id','slug','product_number','list_price','cost_price','sale_price','cover_image_id','images_list','meta_title','meta_keywords','meta_description','short_description','description','ean','type'], 'required'],
            [['categories'], 'safe'],
            [['new', 'featured','visible_individually'], 'default', 'value' => 0],
            [['currency_id'], 'default', 'value' => 1],
            //[['sort'], 'default', 'value' => new Expression("(SELECT MAX(`sort`) as sort FROM products c) + 1")],
        ];
    }

    /**
     * @inheritdoc
     */
    public function ngRestAttributeTypes()
    {
        return [
            'sort' => 'sortable',
            'status' => [
                'selectArray',
                'data' => ProductStatus::listData(),
                'initValue' => ProductStatus::ACTIVE
            ],
            'type' => [
                'selectArray',
                'data' => ProductType::listData(),
                'initValue' => ProductType::BASIC
            ],
            'new' => 'toggleStatus',
            'featured' => 'toggleStatus',
            'visible_individually' => 'toggleStatus',
            'brand_id' => [
                'class' => SelectModel::class,
                'modelClass' => Brands::class,
                'valueField' => 'id',
                'labelField' => 'name'
            ],
            'currency_id' => [
                'class' => SelectModel::class,
                'modelClass' => Currency::class,
                'valueField' => 'id',
                'labelField' => 'title'
            ],
            'name' => 'text',
            'ean' => 'text',
            'upc' => 'text',
            'jan' => 'text',
            'mpn' => 'text',
            'sku' => 'text',
            'product_number' => 'text',
            'inventdim_group_id' => [
                'class' => SelectModel::class,
                'modelClass' => InventDimGroup::class,
                'valueField' => 'id',
                'labelField' => ['code', 'description']
            ],
            'tax_id' => [
                'class' => SelectModel::class,
                'modelClass' => Tax::class,
                'valueField' => 'id',
                'labelField' => 'name'
            ],
            'unit_id' => [
                'class' => SelectModel::class,
                'modelClass' => Units::class,
                'valueField' => 'id',
                'labelField' => 'name'
            ],
            'width' => ['decimal', 'steps' => '0.001'],
            'height' => ['decimal', 'steps' => '0.001'],
            'length' => ['decimal', 'steps' => '0.001'],
            'weight' => ['decimal', 'steps' => '0.001'],
            'cover_image_id' => 'image',
            'images_list' => 'imageArray',
            'list_price' => ['decimal', 'steps' => '0.01'],
            'cost_price' => ['decimal', 'steps' => '0.01'],
            'sale_price' => ['decimal', 'steps' => '0.01'],
            'note' => 'textarea',
            'short_description' => 'textarea',
            'description' => 'wysiwyg',
            'slug' => ['slug'],
            'meta_title' => 'text',
            'meta_keywords' => 'text',
            'meta_description' => 'text',
            'created_at' => 'number',
            'updated_at' => 'number',
            'created_by' => 'number',
            'updated_by' => 'number',
            'deleted' => 'number',
        ];
    }

    /**
     * @inheritdoc
     */
    public function ngRestScopes()
    {
        return [
            ['list', ['cover_image_id','status','type','sort','name','brand_id', 'product_number','currency_id' ,'list_price','sale_price','cost_price']],
            [['create', 'update'], ['currency_id','status','type','name', 'new','unit_id', 'featured', 'visible_individually', 'brand_id','categories' ,'ean','jan','mpn','upc', 'product_number', 'inventdim_group_id', 'tax_id', 'width', 'height', 'length', 'weight', 'images_list', 'cover_image_id', 'list_price','sale_price', 'cost_price', 'note', 'short_description', 'description', 'slug', 'meta_title', 'meta_keywords', 'meta_description']],
            ['delete', false],
        ];
    }

    /**
     * @return array
     */
    public function ngRestAttributeGroups()
    {
        return [
            [['currency_id','list_price','sale_price', 'cost_price', 'tax_id'], 'Price', 'collapsed' => false],
            [['cover_image_id', 'images_list'], 'Media', 'collapsed' => false],
            [['categories'], 'Category and Attributes', 'collapsed' => false],
            [['new', 'visible_individually', 'featured'], 'Marketing', 'collapsed' => false],
            [['unit_id', 'weight', 'width', 'height', 'length'], 'Preferences'],
            [['ean', 'upc', 'jan', 'mpn'], 'Global'],
            [['slug', 'meta_title', 'meta_keywords', 'meta_description'], 'SEO', 'collapsed' => false],
            [['note'], 'Private', 'collapsed' => false],
        ];
    }

    /**
     * @return string[]
     */
    public function extraFields()
    {
        return ['categories'];
    }

    /**
     * @return array
     * @throws \yii\base\InvalidConfigException
     */
    public function ngRestExtraAttributeTypes()
    {
        return [
            'categories' => [
                'class' => CheckboxRelationActiveQuery::class,
                'query' => $this->getCategories(),
                'asArray' => false,
                'labelField' => function($item){
                    return $item->listName;
                },
            ]
        ];
    }

    /**
     * @return string
     */
    public static function sortableField()
    {
        return 'sort';
    }

    /**
     * @return \string[][]
     */
    public function ngRestActiveButton()
    {
        return [
            [
                'class' => 'luya\admin\buttons\ToggleStatusActiveButton',
                'attribute' => 'status',
                'label' => 'Aktif Et',
            ],
        ];
    }

    /**
     * @return array[]
     */
    public function ngRestActiveSelections()
    {
        return [
            [
                'label' => 'Seçilenleri Aktif Et',
                'action' => function (array $items) {
                    /** @var self $item */
                    foreach ($items as $item) {
                        $item->status = ProductStatus::ACTIVE;
                        $item->save();
                    }

                    return true;
                }
            ]
        ];
    }

    /**
     * @return \string[][]
     */
    public function ngRestActiveWindows()
    {
        $configurable = ProductType::getLabel(ProductType::CONFIGURABLE);

        return [
            [
                'class' => ProductDetailActiveWindow::class,
                'label' => 'Product Dimension',
                'icon' => 'info',
                'condition' => "{type}== '{$configurable}'"
            ],
            /*[
                'class' => ProductImageActiveWindow::class
            ]*/
        ];
    }

    /**
     * @inheritDoc
     */
    public function ngRestListOrder()
    {
        return ['sort' => SORT_DESC];
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getImages()
    {
        return $this->hasMany(ProductImages::class,['product_id' => 'id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getVariants()
    {
        return $this->hasMany(ProductVariants::class,['product_id' => 'id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getDefaultVariant()
    {
        return $this->hasOne(ProductVariants::class,['product_id' => 'id'])->andWhere(['default' => 1])->cache(4600);
    }

    /**
     * @return \yii\db\ActiveQuery
     * @throws \yii\base\InvalidConfigException
     */
    public function getColors()
    {
        return $this->hasMany(Color::class, ['id' => 'color_id'])->viaTable(ColorMap::tableName(),
            ['product_id' => 'id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getInventDimGroup()
    {
        return $this->hasOne(InventDimGroup::class, ['id' => 'inventdim_group_id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     * @throws \yii\base\InvalidConfigException
     */
    public function getCategories()
    {
        return $this->hasMany(Category::class, ['id' => 'category_id'])
            ->viaTable(CategoryMap::tableName(), ['product_id' => 'id'])->orderBy('left');
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getBrand()
    {
        return $this->hasOne(Brands::class, ['id' => 'brand_id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     */
    public function getTax()
    {
        return $this->hasOne(Tax::class,['id' => 'tax_id']);
    }

    /**
     * @return \yii\db\ActiveQuery
     * @throws \yii\base\InvalidConfigException
     */
    public function getSizes()
    {
        return $this->hasMany(Size::class, ['id' => 'size_id'])->viaTable(SizeMap::tableName(), ['product_id' => 'id']);
    }
}

ProductWidget

public function run()
{
    if($this->product instanceof Products){
        $this->_model = $this->product;
    }else{
        $this->_model = Products::findOne(['id' => $this->product]);
    }

    echo $this->render('_product',[
        'model' => $this->_model,
        'image_filter' => $this->image_filter
    ]);
}

SearchModel

$query = Products::find()->with(['defaultVariant','tax'])
            ->andWhere(['deleted' => 0]);

$dataProvider = new ActiveDataProvider([
    'query' => $query,
    'key' => function ($model) {
        return YII_ENV_PROD ? md5($model->id) : $model->id;
    },
    'pagination' => [
        'pageSize' => 21
    ],
    'sort' => [
        'enableMultiSort' => false,
        'defaultOrder' => [
            'sort' => SORT_ASC,
        ],
        'attributes' => [
            'sort' => [
                'asc' => ['sort' => SORT_ASC],
                'desc' => ['sort' => SORT_DESC]
            ],
            'price' => [
                'asc' => ['sale_price' => SORT_ASC],
                'desc' => ['sale_price' => SORT_DESC],
                'default' => SORT_DESC,
                'label' => 'Fiyata göre artan',
            ],
        ],
    ],
]);

[nadar]

Do you display relations? Are you sure its the log behavior? (if you just comment out the log behavior, does it change)? I think this could be because of SelectModel plugin.

1. please uncomment the log behavior, and let me know if this is the performance issue
2. please try to use SelectRelationActiveQuery instead of SelectModel => https://luya.io/guide/ngrest-plugin-select
3. also make sure to preload your relation data (if needed).

Yes NgRestModel does provided some functions and helpers which requires more memory then ActiveRecord, but it should not be sooo drastically.

What you also could do of course, is just use the ngrest model in admin area context, and generate an active record only model for frontend, but i think it would be better to find that memory pit.

Thanks for helping us making LUYA better