Webklex / laravel-imap

Laravel IMAP is an easy way to integrate both the native php-imap module and an extended custom imap protocol into your Laravel app.
https://www.php-imap.com
MIT License
641 stars 182 forks source link

Chunk with 0 messages #438

Open bilogic opened 2 years ago

bilogic commented 2 years ago

Describe the bug Chunk returns with 0 messages when there are still >1000 messages on server

To Reproduce Steps to reproduce the behavior: Connecting to gmail, $chunk_size and $start_chunk set to 1 The issue remains even if $chunk_size is set to 10

$i  = 0;
$folder->query()->all()->chunked(function ($messages, $chunk) use (&$i) {
    echo "- chunk #$chunk, ".$messages->count().' messages';
    $messages->each(function (Message $message) use (&$i) {
        file_put_contents(storage_path("messages/$i.xml"), $message->getRawBody());
        $i++;

        if ($message->move('INBOX/Downloaded') == true) {
            echo "- Moved {$message->uid}").PHP_EOL;
        } else {
            echo "- Cannot move {$message->uid}".PHP_EOL;
        }
    });
}, $chunk_size = 1, $start_chunk = 1);

Expected behavior There should always be >0 messages in each chunk, never 0 messages.

Screenshots

- chunk #1, 1 messages
- Moved 6003
- chunk #2, 1 messages
- Moved 11
- chunk #3, 1 messages
- Moved 20
- chunk #4, 1 messages
- Moved 27
- chunk #5, 1 messages
- Moved 31
- chunk #6, 1 messages
- Moved 35
- chunk #7, 1 messages
- Moved 38
- chunk #8, 1 messages
- Moved 40
- chunk #9, 1 messages
- Moved 42
- chunk #10, 1 messages
- Moved 44
- chunk #11, 1 messages
- Moved 46
- chunk #12, 1 messages
- Moved 48
- chunk #13, 0 messages
- chunk #14, 0 messages
- chunk #15, 0 messages
- chunk #16, 0 messages
- chunk #17, 0 messages
- chunk #18, 0 messages
- chunk #19, 0 messages
- chunk #20, 0 messages

Desktop / Server (please complete the following information):

Additional context

  1. I want to download and save raw messages one by one without it becoming a memory hog.
  2. Once I kill the script and restart, it downloads another 12 to 13 messages before returning 0 messages again, so it doesn't seem to be a rate limiting issue
Tahiaji commented 1 year ago

The problem is: when you move email - first item from collection removed too. Its like iterate array and remove elements.

Workaround for me - iterate 1 by 1, and request every time first element

/** @var WhereQuery $query */
$totalMessages = $this->inboxFolder->messages()->all()->count();
for ($i = 1; $i <= $totalMessages; $i++) {
   /** @var MessageCollection $messages */
   $messages = $this->inboxFolder->messages()->all()->limit(1, 1)->get();
   foreach ($messages as $message) {
      $message->move('...'); // and so on
   } 
}