lk-geimfari / mimesis

Mimesis is a robust data generator for Python that can produce a wide range of fake data in multiple languages.
https://mimesis.name
MIT License
4.39k stars 330 forks source link

Slow execution speeds and repeated output #286

Closed jeffgardnerdev closed 6 years ago

jeffgardnerdev commented 6 years ago

Providers utilizing the utils.custom_code method perform slowly and output the same value several times when called repeatedly from the same program.

To replicate, update the comp.py file by replacing p.full_name() with p.telephone('###-###-####') on line 27. If you change the count to something small, say 50, and print out the output of p.telephone('###-###-####'), you'll see that many of the generated phone numbers are identical.

lk-geimfari commented 6 years ago

@gardnerjh The function custom_code works really really slow and we need to fix it as soon as possible.

Concerning telephone(), then i cant replicate this issue, because in my case everything is okay:

>>> from mimesis import Personal
>>> 
>>> p = Personal('ru')
>>> 
>>> 
>>> def generate_mimesis(count):
...     db = [p.telephone('###-###-####') for _ in range(0, count)]
...     return db
... 
>>> 
>>> result_1 = generate_mimesis(50)
>>> result_2 = generate_mimesis(500)
>>> result_3 = generate_mimesis(5000)
>>> result_4 = generate_mimesis(50000)
>>> 
>>> len(set(result_1))
50
>>> len(set(result_2))
500
>>> len(set(result_3))
5000
>>> len(set(result_4))
50000

All the results are unique, as you can see. I have not idea why it's works differently in your case.

Which version of library do you use? What is your OS?

jeffgardnerdev commented 6 years ago

Here are my specs: Mimesis 0.0.10 Python 3.6.3 Mac OS High Sierra (10.13)

I copied your code and ran it and it comes back with stuff like this:

>>> len(set(result_1))
1
>>> len(set(result_2))
4
>>> len(set(result_3))
8
>>> len(set(result_4))
10
lk-geimfari commented 6 years ago

@gardnerjh How do you installed mimesis? Using pip? Try installing manually by copying the repository.

We are preparing to release the first major version (1.0.0). It's already done, but i have not published it on PyPi and it's possible that this issue is only in 0.0.10.

jeffgardnerdev commented 6 years ago

Yeah I did use pip. I'll try a manual install and report back.

lk-geimfari commented 6 years ago

@gardnerjh Okay. Thanks!

lk-geimfari commented 6 years ago

@sobolevn It seems there is a problem with the Mac, but i'm not sure. Can you check code above? It's works as expected in your case?

sobolevn commented 6 years ago

I can confirm this. Happening on my mac.

>>> from mimesis import Personal
>>> p = Personal()
>>> db = [p.telephone('###-###-####') for _ in range(0, 5000)]
>>> set(db)
{'281-313-7449', '270-378-9364', '657-361-1524', '344-172-4776'}
lk-geimfari commented 6 years ago

@sobolevn I can't understand why. On my Ubuntu 16.04 everything is okay. Any ideas?

image

sobolevn commented 6 years ago

That's how Code works. Sorry for this long snippet.

>>> from mimesis import Code
>>> Code().custom_code('+#-(###)-###-####')
'+8-(809)-239-7427'
>>> Code().custom_code('+#-(###)-###-####')
'+5-(422)-187-7212'
>>> Code().custom_code('+#-(###)-###-####')
'+8-(866)-394-4099'
>>> Code().custom_code('+#-(###)-###-####')
'+8-(866)-394-4099'
>>> Code().custom_code('+#-(###)-###-####')
'+5-(422)-187-7212'
>>> Code().custom_code('+#-(###)-###-####')
'+3-(441)-724-7760'
>>> Code().custom_code('+#-(###)-###-####')
'+7-(701)-377-1367'
>>> c = Code()
>>> c.custom_code('+#-(###)-###-####')
'+8-(809)-239-7427'
>>> c.custom_code('+#-(###)-###-####')
'+7-(481)-381-7403'
>>> c.custom_code('+#-(###)-###-####')
'+1-(156)-511-0809'
>>> c.custom_code('+#-(###)-###-####')
'+1-(944)-188-9571'
>>> c.custom_code('+#-(###)-###-####')
'+0-(269)-069-0118'
>>> c.custom_code('+#-(###)-###-####')
'+4-(391)-429-9055'
>>> c.custom_code('+#-(###)-###-####')
'+7-(839)-911-3901'
>>> c.custom_code('+#-(###)-###-####')
'+1-(261)-055-3734'
>>> c.custom_code('+#-(###)-###-####')
'+7-(433)-762-9017'
>>> c.custom_code('+#-(###)-###-####')
'+0-(837)-212-7688'

Here's the reason behind this error: everytime we create a new instance of Code it only has some values to generate. So, it generates only them.

But with the same instance it works fine.

lk-geimfari commented 6 years ago

@sobolevn So, version 1.0.0 should work correctly, because i have fixed this issue a few days ago, here: utils.custom_code.

sobolevn commented 6 years ago

Yeap, it really should. @gardnerjh thanks for this issue, it is already fixed in the newest version.