statement separator don't treat comments as a token separator

[apstndb]

The statement separator implementation treat comments as empty but it should be treated as a kind of a token separator as like whitespaces. This behavior can make query execution an error or even change the meaning of the query.

Example

spanner-cli evaluate SELECT 0x1/**/A as SELECT 0x1A.

$ spanner-cli -d ${DATABASE} --execute='SELECT 0x1/**/A' -t                                             
+----+
|    |
+----+
| 26 |
+----+

Expected behavior

It should be evaluated as SELECT 0x1 A(equivalent of SELECT 0x1 AS A).

$ gcloud spanner databases execute-sql ${DATABASE} --sql='SELECT 0x1/**/A'               
A
1

Related specification

https://cloud.google.com/spanner/docs/reference/standard-sql/lexical?hl=en

You can separate tokens with comments or whitespace such as spaces, backspaces, tabs, or newlines.

[apstndb]

single-line comments seems to have a same problem.

spanner> SELECT 0x1--
      -> A;
+----+
|    |
+----+
| 26 |
+----+
1 rows in set (0.95 msecs)

spanner> SELECT 0x1#
      -> A;
+----+
|    |
+----+
| 26 |
+----+
1 rows in set (2.09 msecs)