fukamachi / cl-dbi

Database independent interface for Common Lisp
202 stars 28 forks source link

MySQL memory leak #73

Open simon639 opened 2 years ago

simon639 commented 2 years ago
(let* ((pq (dbi:prepare conn "SELECT * FROM visit"))
     (query (dbi:execute pq nil)))
       (dbi:fetch-all query))

above code run in hunchentoot will lead to memory leak in MySQL

https://blog.actorsfit.in/a?ID=01800-28830ddf-a436-4fff-b97d-d952487c8948

the result set of mysql_store_result(), mysql_use_result(), and mysql_list_dbs() must call mysql_free_result() to release the memory used by the result set after completing the operation on the result set.

(defmethod execute-using-connection ((conn dbd-mysql-connection) (query dbd-mysql-query) params)
  (let* (took-usec
         (result
           (with-error-handler conn
             (with-took-usec took-usec
               (query (funcall (query-prepared query) params)
                      :database (connection-handle conn)
                      :store nil)))))
    (return-or-close (owner-pool result) result)
    (next-result-set result)
    (cond
      ((mysql-use-store query)
       (multiple-value-bind (rows count)
           (fetch-all-rows result)
         **(cl-mysql-system::mysql-free-result (cl-mysql-system::result-set result))**
         (sql-log (query-sql query) params count took-usec)
         (setf result (make-mysql-result-list rows count))
         (setf (query-row-count query) count)))
      (t
       (sql-log (query-sql query) params nil took-usec)))
    (setf (query-results query) result)
    query))
fukamachi commented 2 years ago

Confirmed.

It seems MySQL driver has no finalization process while PostgreSQL driver has finalization on GC (powered by trivial-garbage) (SQLite3 has a function free-query-resources to free it manually) Need to be fixed.

fukamachi commented 2 years ago

It seems return-or-close does free the last result set internally. https://github.com/hackinghat/cl-mysql/blob/3fbf6e1421484f64c5bcf2ff3c4b96c6f0414f09/pool.lisp#L256

As a result, your single line patch raises free(): double free detected in tcache 2. Requires more investigation.