Counting items, and also iterating over them

[geeflaxx]

Hey Creater,

the library is amazing! I have a bit trouble with PyCharm, but in the Terminal it looks great.

I wanna count the lines of my text file and give it to the total result in the alive_bar, sadly I got this error: |⚠︎ | (!) 0/3 [0%] in 0.0s (0.00/s)

it works if I use a number. What shall I do?

with open("listofsites.txt", "r") as file:
    lines = sum(1 for line in file)
    with alive_bar(lines) as bar:
        for hostname in file:
            bar()
            time.sleep(1)

[TheTechRobo]

The warning sign means the bar didn't finish incrementing (i.e. the total hasn't been filled). In this case it's because when you finish iterating over the lines of the file with lines = sum(1 for line in file), the iterator is exhausted. What I mean is that it reaches the end of the file, so you're asking it to go through every hostname in the file starting at the end. There are none left, so nothing in for hostname in file: gets executed.

[geeflaxx]

Thanks!

Do you have a recommandation for me?

[TheTechRobo]

I'm not actually sure how to fix it. You might have to open the file twice - once to get the amount of lines, the other to use them. Won't be pretty, but it'll work...

Maybe there's a solution I'm missing though

[rsalmei]

Hi @geeflaxx, thanks!

Yeah, when you did sum(1 for line in file), you actually consumed the whole file... This is what Python is seeing:

In [7]: with open('requirements/test.txt') as file:
   ...:     print(sum(1 for line in file))
   ...:     print(sum(1 for line in file))
   ...:
6
0

Unfortunately, if you need to count items in an iterator, and also iterate over the same items, you have to either:

• read the file again, as @TheTechRobo has suggested;
• or just store it, if the file is not that big. I recommend the latter, since usually memory is cheap, while I/O is not.

Something like this:

In [10]: with open('requirements/test.txt') as file:
    ...:     lines = [line for line in file]
    ...:
    ...:     print(sum(1 for line in lines))
    ...:     print(sum(1 for line in lines))
    ...:
    ...:     with alive_bar(len(lines)) as bar:
    ...:         for line in lines:
    ...:             bar()
    ...:
6
6
|████████████████████████████████████████| 6/6 [100%] in 0.0s (227/s)

Closing this one, but feel free to reply if you want!

[TheTechRobo]

With @rsalmei's suggested code, I'd do len(...) instead of sum(1 for line in lines)

[rsalmei]

Yes, exactly. The sums are for demonstration purposes only. So he sees that now we can consume the iterators more than once... Note I did use len in alive_bar 😉