shetabit / visitor

a laravel package to work with visitors and retrieve their informations
MIT License
527 stars 68 forks source link

When large data can slow down my website, is this the effect of log duplication ? #61

Closed gagaltotal closed 2 months ago

gagaltotal commented 2 months ago

Is this duplication of data logs the cause? I have tested various queries using Selectraw which are pruned daily and monthly when calculating when this data is a lot, it really affects the slowness of the website and the database in MySQL, because it has accommodated 700K data in 3 years

please help if there is an update for duplicate logs?

example code daily in my helpers :

if (!function_exists('getVisitorToday')) {
    function getVisitorToday()
    {
        $result = DB::table('shetabit_visits')->selectRaw('COUNT(DISTINCT ip) as count, DATE(created_at) as date')
                ->groupByRaw('DATE(created_at)')
                ->whereDate('created_at', now()->today())
                ->first();
        if ($result) {
            return $result->count;
        } else {
            return '';
        }
    }
}

example code monthly in my helpers :

if (!function_exists('getVisitorMonth')) {
    function getVisitorMonth()
    {
        $result = DB::table('shetabit_visits')->selectRaw('COUNT(DISTINCT ip) as count, MONTH(created_at) as month')
                ->groupByRaw('MONTH(created_at)')
                ->whereYear('created_at', date('Y'))
                ->whereMonth('created_at', date('m'))
                ->limit('20000')
                ->first();
        if ($result) {
            return $result->count;
        } else {
            return '';
        }
    }
}
gagaltotal commented 2 months ago

I made a view query in MySQL and it also didn't affect the speed of the website, because there was too much data from duplicate logs

example query view days :

CREATE VIEW visitor_day_counts AS
SELECT 
    COUNT(DISTINCT ip) as visitor_count,
    DATE(created_at) as visitor_day
FROM 
    shetabit_visits
GROUP BY 
    DATE(created_at);

example query month :

CREATE VIEW visitor_month_counts AS
SELECT 
    COUNT(DISTINCT ip) as visitor_count,
    DATE_FORMAT(created_at,'%Y-%m') AS visitor_month
FROM 
    shetabit_visits
GROUP BY 
    DATE_FORMAT(created_at,'%Y-%m');
gagaltotal commented 2 months ago

SOLVED DUPLICATE LOG :

            $visit = ViewPage::where('ip', $request->ip())->count();

            if ($visit >= 1) {
                return view(getTheme('home'));
            }else{
                visitor()->visit(); // create a visit log
                return view(getTheme('home'));
        }

File Shetabit/visitor/src/Visitor.php :

    public function visit(Model $model = null)
        {
            if (in_array($this->request->path(), $this->except)) {
                return;
            }

            $data = $this->prepareLog();

            $existingVisit = Visit::where('ip', $data['ip'])
                ->where('url', $data['url'])
                ->where('device', $data['device'])
                ->where('platform', $data['platform'])
                ->first();

            if ($existingVisit) {
                return $existingVisit;
            }

            if (null !== $model && method_exists($model, 'visitLogs')) {
                $visit = $model->visitLogs()->create($data);
            } else {
                $visit = Visit::create($data);
            }

            return $visit;
        }