fzaninotto / Faker

Faker is a PHP library that generates fake data for you
MIT License
26.78k stars 3.59k forks source link

Allowed memory exhausted. #1125

Closed mikeshow closed 5 years ago

mikeshow commented 7 years ago

Here what I get when I use realText() in a loop:

PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 4096 bytes) in C:...\faker\fzaninotto\faker\src\Faker\Provider\Text.php on line 91

fzaninotto commented 7 years ago

What's your locale? How many iterations have you done when the memory is exhausted?

mikeshow commented 7 years ago

Local fr_FR and two successful and then error.

mikeshow commented 7 years ago

I tried to place the call into a function hope the memory would be freed upon return, but changed nothing.

curry684 commented 7 years ago

You could try invoking gc_collect_cycles at the start of the function to force cleanup of any pending cycles left after the previous invocation.

Not a solution but a workaround of course.

mikeshow commented 7 years ago

Thank you very much, apparently that do the trick.

curry684 commented 7 years ago

@fzaninotto perhaps if the text generator is that memory hungry this should be part of the provider wrapper?

cristiangrama commented 7 years ago

Got same issue here in vendor/fzaninotto/faker/src/Faker/Provider/Base.php.

mohamednizar commented 7 years ago

I fixed this issue temporarily by extending the php memory size.

c0nst4nt commented 7 years ago

I'm sorry, but why it's closed? Is it fixed?

mikeshow commented 7 years ago

Hi, I closed it because I got an answer which fixes my problem. Now I don't know if @fzaninotto has or is about to fix it.

c0nst4nt commented 7 years ago

Okey, got it. Increasing memory helped me too. But it's interesting, am I wrong with using it or it's a bug connected with memory leaks?

prazian commented 7 years ago

Increasing memory doesn't help me if I run unit test more than 3-4 times, so I had to manually add gc_collect_cycles() before each call to text generating method.

Is there any solution for this issue? :)

curry684 commented 7 years ago

Like I said in https://github.com/fzaninotto/Faker/issues/1125#issuecomment-268825965 I'd recommend adding the gc_collect_cycles call to the wrapper.

Lloople commented 6 years ago

Ran into this problem. Changed $faker->realText(100) to $faker->sentence(10). Getting similar fake data but without error (Can't be sure if I'll get the error in a future when adding more tests tho)

Tbaile commented 6 years ago

Happened again, it appears that the method that allocates the realText method (or others) tries to allocate a huge memory (library of all possible text, maybe?) that isn't released when a first generation is done. Changed as @Lloople says fixes it temporarily. Adding a clear to the memory is a good way to go, still odd for the software acting like this.

pejaycz commented 6 years ago

Hi, I have the same problem with realText (cs_CZ locale). Every call realText() takes almost 30MB of RAM (7000 lines of text in cs_CZ localization) causing memory exception after few calls. I have done few experiments with $indexSize and gc_collect_cycles(), but nothing worked for me.

Only solution is change from realText() to sentences(), but generated text is not localized.

Maybe "Markov chain algorithm" is too complex for "fake" data?

pimjansen commented 5 years ago

I tried running it in a loop but there is no such thing as increasing memory or a leak. The usage is stable at MEM: 38.50390625MB

$faker = Faker\Factory::create( 'cs_CZ' );
for ($i=0;$i<1000;$i++) {
    $faker->realText(10000).PHP_EOL;
}

Please note that you should not create a new Faker instance on the factory for each loop which will consume a lot of memory as intended where it will keep it all in memory.

pimjansen commented 5 years ago

I will close this issue for now since the fact that we can not reproduce the issue and it is pretty old already. In case more is needed, just let me know

bytestream commented 4 years ago

@pimjansen any comments on https://github.com/fzaninotto/Faker/pull/1730#issuecomment-619435477 ?

aurakarya commented 4 years ago

it in a loop but there is no such thing as increasing memory or a leak. The usage is stable at MEM: 38.50390625MB

tried running the loop 1.000.000 and got the memory exhausted in a loop of about less than 200.000. seems like you running it in a very short loops.

just fyi, factory created at outside of the loop. even after i called gc_collect_cycles() at each loop still have the same error.