Not able to limit the memory usage of the VW process

[Casperfrc]

Describe the bug

When training a model to play a game of hangman I have issues when upping the amount of iterations that the model will train on the data. It is very clearly an issue of running out of memory, but I am simply cannot seem to limit the amount of memory the process uses and I am not sure if it is me doing something wrong.

To Reproduce

I have used this example as my key source of inspiration: https://vowpalwabbit.org/tutorials/cb_simulation.html#first-scenario

With that I have created the following simulation loop `def run_simulation(vw, num_iterations, words_of_the_game, actions, cost_function, do_learn = True): cost_sum = 0.

for i in range(1, num_iterations+1):
    word_of_the_game = choose_word_of_the_game(words_of_the_game)
    current_word_state = " " * len(word_of_the_game)
    context = {'word_of_the_game': word_of_the_game, 'current_word_state': current_word_state}

    while(context['word_of_the_game'] != context['current_word_state']):      
        action, prob = get_action(vw, context, actions)            

        word, cost = cost_function(context, action)
        context['current_word_state'] = word
        cost_sum += cost

        if do_learn:
            vw_format = vw.parse(to_vw_example_format(context, actions, (action, cost, prob)),pyvw.vw.lContextualBandit)
            vw.learn(vw_format)`

Basically what happens is that a random word of no more than 9 characters is picked and then the model guesses one of the 26 characters in the English alphabet until it gets the complete word.

I can run it with 10, 25 and 50 iterations, but as I set it to a 100, my memory and then swap caps out and my bash shell shuts it down.

I run the script with the following command: systemd-run --scope -p MemorySwapMax=500M -p MemoryMax=5000M python3 hangman_attempt/test.py

Expected behavior

I might be wrong in expecting the memory limits to work like that? Is there another way to limit it through the framework that I have not found?

I know it could be optimised to not run only single core, but memory would become an issue at some point anyhow.

Observed Behavior

VW doesn't do anything specific in this case. My bash shell shuts it all down.

Environment

VW 8.8.1 installed through pip with Python 3.6.9. I am running on Ubuntu 18.04

Additional context

I'm sorry if this has nothing to do with the kinds of issues you guys usually help with!

[jackgerrits]

If you are using the parse function then you also need to pass those examples to finish_example. Could you try updating this section of the code and let me know if this fixes your issue.

if do_learn:
    vw_format = vw.parse(to_vw_example_format(context, actions, (action, cost, prob)),pyvw.vw.lContextualBandit)
    vw.learn(vw_format)
    vw.finish_example(vw_format)

Edit: just realized this is an issue in the tutorial. I'll get that fixed up too. Thanks for reporting this!

[Casperfrc]

Perfect @jackgerrits, this made me able to not only run it with a 100 iterations but also with 5000 iterations and I wasn't close to running out of memory a single time.

Thanks a tonne. :)