django / djangoproject.com

Source code to djangoproject.com
https://www.djangoproject.com/
BSD 3-Clause "New" or "Revised" License
1.86k stars 938 forks source link

bulk_create example code provided seems to go into infinite loop #1449

Closed blokeish closed 4 months ago

blokeish commented 6 months ago

The code provide here seems to go into an infinite loop. The batch list variable seems to always get the first 100 items from objs list, so the batch list will never go empty.

from itertools import islice

batch_size = 100
objs = (Entry(headline="Test %s" % i) for i in range(1000))
while True:
    batch = list(islice(objs, batch_size)) # <--- this always seems to be yielding the same single first batch.
    if not batch:
        break
    Entry.objects.bulk_create(batch, batch_size) 

I have seen this had got carried over to stack over flow. I am relatively new to Python and Django, so, sorry if I am making a stupid entry.

MrAllenA commented 6 months ago
from itertools import islice
from myapp.models import Entry 

batch_size = 100

while True:
    objs = (Entry(headline="Test %s" % i) for i in range(1000))

    batch = list(islice(objs, batch_size))
    if not batch:
        break
    Entry.objects.bulk_create(batch, batch_size)

try this

lachlantownshend commented 6 months ago

Make sure that the objs = (Entry(headline="Test %s" % i) for i in range(1000)) line is making a tuple rather than a list. If using a list it won't progress the iterator and will loop infinitely

felixxm commented 4 months ago

This is the issue tracker for the djangoproject.com website. Please see TicketClosingReasons/UseSupportChannels for ways to get help with Django usage.