This PR migrates the implementation of the grammar engine to an outside library XGrammar. The implementation of XGrammar is basically the same as our current version, but more modularized and robost. XGrammar will be continuously enhanced in the future.
This PR migrates the implementation of the grammar engine to an outside library XGrammar. The implementation of XGrammar is basically the same as our current version, but more modularized and robost. XGrammar will be continuously enhanced in the future.
See https://github.com/mlc-ai/xgrammar for more details.
cc @tqchen @MasterJH5574 @cyx-6 @CharlieFRuan