How to tokenize C code using pycparser?

[xu-zhiwei]

For example, if I have a C code snippet:

int main() {
        int i = 0;
        return 0;
}

How to tokenize this code snippet and get the result like:

['int', 'main', '(', ')', 'int', 'i', '=', '0', ';', 'return', '0', ';', '}']

[eliben]

Use the CLexer class from https://github.com/eliben/pycparser/blob/master/pycparser/c_lexer.py#L15

You can use the tests as inspiration: https://github.com/eliben/pycparser/blob/master/tests/test_c_lexer.py