gpslab / geoip2

A Symfony Bundle for the Maxmind GeoIP2 API
https://www.maxmind.com/
MIT License
54 stars 12 forks source link

Out of memory updating GeoLite2-City #82

Closed mreiden closed 3 years ago

mreiden commented 3 years ago

Since July 22nd, geoip2:update runs out of memory when updating the GeoLite2-City edition because the current memory usage exceeds the default 128MB memory limit on Fedora:Memory Usage: 21795872 Memory Peak Usage: 146951168 https://github.com/gpslab/geoip2/blob/5710d126645a1d778ad61b70d5d5744b9d31a229/src/Downloader/MaxMindDownloader.php#L75

Two solutions:

  1. Add ini_set('memory_limit', '256M'); before opening the archive with PharData. Not a great solution in my opinion.
  2. Use splitbrain/php-archive to reduce memory usage from 147MB to 18MB: Memory Usage: 17999088 Memory Peak Usage: 18098424 by replacing

https://github.com/gpslab/geoip2/blob/5710d126645a1d778ad61b70d5d5744b9d31a229/src/Downloader/MaxMindDownloader.php#L74-L85

with:

use splitbrain\PHPArchive\Tar;
...
        // extract tar.gz archive
        $tar = new Tar();
        $tar->open($tmp_zip);
        $tar->extract($tmp_untar);
        $tar->close();
        unset($tar);

        $this->logger->debug(sprintf('Tar archive extracted to %s', $tmp_untar));
peter-gribanov commented 3 years ago

Fixed by #83