Persisting autocomplete object - TypeError: can't pickle _thread.lock objects

[hblauth]

I am using this library in a lambda function for autocomplete in a search box.

A POST request is sent with the search string to the lambda function, which returns the top 10 alphabetical matches from the DWG. (I will eventually use count when I have figured out a method of ranking the responses).

To speed things up, I want to build the autocomplete object, store it in s3, and then load it in the lambda function instead of having to build the dwg each time.

When I try to pickle the autocomplete object, I get this error message: "TypeError: can't pickle _thread.lock objects"

I think it's something to do with multiprocessing, lock, queue or logger, but I have no experience with any of that stuff so I'm at a loss.

Is there a way to get round this - either by persisting the autocomplete in a different way, or making it pickle-able?

Thanks

[seperman]

Interesting. What you could do is to pickle self._dwg because that's the graph once populated. Then instantiate AutoComplete with an empty dictionary or words. Then overwrite self._dwg

So:

# ac is the autocomplete object.
dump = pickle.dumps(ac)

# Load it via lambda
ac = Autocomplete(words={})
ac._dwg = pickle.loads(dump)

I think you should get a better performance if you have a server that runs autocomplete and the edge nodes call that service to get the results.

[seperman]

Hi @hblauth Did that work?

[hblauth]

Yes! Thanks very much.