search

vnmakarov / yaep

Yet Another Earley Parser
Other
135 stars 13 forks source link

Make libyaep reentrant #12

Open TheCount opened 5 years ago

TheCount commented 5 years ago

YAEP currently manipulates a lot of static variables during its operation. This makes it impossible to use multiple parsers asynchronously.

One simple preliminary fix would be to declare all these variables thread_local. While this does not make libyaep reentrant, it would at least become possible to use multiple parsers concurrently, as long as each parser has a thread of its own and stays on that thread. This preliminary fix would, strictly speaking, break the API, though, because it would no longer be possible to use the same parser from different threads.

The better fix would be to make libyaep fully reentrant by eliminating all static variables. Unless extra parameters are required for this for exposed functions, this could be implemented without breaking the API. Parsers could then be used asynchronously and, as long the caller adheres to a few simple rules, would also be thread-safe.

vnmakarov commented 5 years ago

Thank you for raising the issue.

Basically YAEP was written about 20 years ago and was a part of COCOM tool set. Concurrency was not a mainstream that time.

I guess it is possible to make YAEP thread-safe by adding global vars to grammar object. If it is not true, new thread-safe functions could be a solution to save API compatibility. On the other hand, I don't think YAEP is widely used to be worry about breaking API.

Unfortunately, I am too busy to address this issue (these days I have many other projects to work on).

TheCount commented 5 years ago

Understood. I'll give it a shot, but my time is also limited, so I can't make any promises.

In any case, thanks for writing YAEP.

TheCount commented 5 years ago

Just a heads-up: the allocator used by YAEP is not thread-safe/reentrant by itself. So I'll try to fix the allocator first, then YAEP.