Issue with .save() method on NeuralLinear model

[AlliedToasters]

First, I'd like to congratulate you for such a nice algorithm, I've been rading it about these days and the results found in mentioned paper look nice, as well as the contextualization for many problems. In my case, I've trained and tested a 100k dataset here and it kind of worked (low accuracy/recall), but as you've mentioned in your demos notebooks, it is indeed a more difficul task. My question is: is there any special care about saving results? When I try to use the save() method, my machine breaks. I've used the pre-defined hyperparameters for the Neuro Linear. Thanks!

Originally posted by @lgabs in https://github.com/fellowship/space-bandits/issues/6#issuecomment-496696351

[AlliedToasters]

@lgabs thanks for your feedback, I'm glad you've been able to get some results using this implementation!

Could you please share some more information about your issue? Copy/paste your code and the traceback. Information about your system and environment would be helpful as well.

Thank you!

[lgabs]

I am pretty much following your demo approach, the training worked and scoring too. The problem is that I do not get a traceback, since my machine stops functioning and I am obliged to restart it. The following code resumes what I was doing here. The context here is:

_In a cal center environment, at each trial t, the algorithm sees a customer with context Xt and decides whether to call him or not (two actions). Then, it sees the reward associated with the success of converting a sale (r = revenue - call costs in the action of calling and r = revenue when the conversion occurred with the decision of not calling). The optimal policy is not know, and I thought of using the algorithm as a better choice than A/B testings.

thompson_model_v2_copy.pdf

[AlliedToasters]

@lgabs , thanks for the additional information.

The problem stems from the serialization procedure (pickle), which is honestly a poor choice. I believe this is due to the large amount of data you have in your model. Pickle probably uses too much memory or something like that.

In a future version, the model will use a pandas dataframe to handle data (instead of torch tensors). During save, the contents will be written out to a standalone file and the model itself will be pickled, and all the resulting data will be zipped up (or something like this), which should save your system memory and succeed. I can't say when this feature will be finished.

In the meantime, try limiting the number of examples you use in training or playing with the memory_size parameter to limit the number of examples the model remembers.

Finally, an AUC of less than .5 indicates there's something wrong with your evaluation. See this response.

[lgabs]

Thank you very much! It's probably the problem you mentioned with memory size. I'll try fewer examples and post the answers here, and thanks for the AUC tip, I'll dive into that!