Track complete parsing support in Vitess

GuptaManan100 commented 3 years ago

Description

Vitess has its own in-built SQL-parser which it uses to understand the query and represent as structs for further processing. As of now, a lot of MySQL structs are not parsed and result in syntax errors. This issue is to track the progress on adding parsing for such constructs

[x] Partitions
- [x] #9682
- [x] #9696
- [x] Partitiondefinition, sub partitions
[x] Prepare
[x] Collations
[x] Charsets
[ ] Functions (MySQL 5.7)
[ ] Functions (MySQL 8.0)
[ ] String functions (https://dev.mysql.com/doc/refman/8.0/en/string-functions.html)
[x] Date and time literals (https://dev.mysql.com/doc/refman/8.0/en/date-and-time-literals.html)
[ ] #4099
[x] #4347
[x] #4038
[ ] https://dev.mysql.com/doc/refman/8.0/en/spatial-analysis-functions.html
[x] https://github.com/vitessio/vitess/issues/9716
[x] https://github.com/vitessio/vitess/issues/9951
[ ] https://github.com/vitessio/vitess/issues/10760

Work Done:

8797
8821
8884
8918
9006
9011
9029
9075
8691

ritwizsinha commented 3 years ago

Hello @GuptaManan100 I have been trying out and understanding various projects in CNCF ecosystem and I am particularly interested in those related to databases and networking so I would like to get involved with this project and work on this issue. I would like to contribute to this project and increase my knowledge. I would be familiarising myself with terminology and get to know better about Vitess in the coming days. Any help appreciated.

GuptaManan100 commented 3 years ago

@ritwizsinha good to hear! This project is part of the LFX program. More information at https://github.com/cncf/mentoring/tree/main/lfx-mentorship. We would love to see you apply in it. All the best!

ritwizsinha commented 3 years ago

Thanks @GuptaManan100 I would like to apply to it, what all do I need to provide to improve my applications, do I need to have previous knowledge of Vitess, or previous contributions in this repository to get selected? Do I have to write out an implementation proposal for this feature(I am asking because I heard that every org did it differently)? What all would you like to see in the application ?

GuptaManan100 commented 3 years ago

@ritwizsinha No, we do not expect any previous knowledge of Vitess from the applicants. There is no explicit requirement for a implementation proposal either, but it would be great if you would be able to do a gap analysis of what all Vitess parser is missing from MySQL. I have written some of the constructs that I know that Vitess does not parse correctly

derekperkins commented 3 years ago

There are still a significant number of 8.0 functions without support: https://github.com/vitessio/vitess/issues/4099

ritwizsinha commented 3 years ago

Okay thanks @GuptaManan10 and @derekperkins I will look into it

aribalam commented 3 years ago

Hi @GuptaManan100 , a student from IIT Kharagpur here. I would like apply for this project under the LFX mentorship program. I have read the previous comments and the requirements you mentioned. Looking forward to it. :)

ritwizsinha commented 3 years ago

@GuptaManan100 I have read bit of the documentation of Vitess and set it up locally using Docker, now to identify which constructs are missing, I am thinking of trying to execute the methods you mentioned above, for example for partitioning. then for collation etc. Is their any document which specifies what all the Vitess parser recognizes and parses or we have to check it all manually?

GuptaManan100 commented 3 years ago

@ritwizsinha Have a look at https://github.com/vitessio/vitess/blob/main/go/vt/sqlparser/sql.y. This file is our yacc parser configuration. So this is the authoritative source on what we parse and what we don't. Within the same package we have a parse_test.go file which has the parsing tests.

ritwizsinha commented 3 years ago

@GuptaManan100 I wrote a quick script to match all the functions in here https://dev.mysql.com/doc/refman/8.0/en/built-in-function-reference.html which aren't present in the https://github.com/vitessio/vitess/blob/main/go/vt/sqlparser/sql.y. using grep and it turns out that 340/482 of those functions were not present in the yacc parser config. I may be wrong because I am not an expert in bash scripts but I searched some and they didn't exist

``` >> >= <> << <= <=> -> ->> ABS ACOS ADDDATE ADDTIME AES_DECRYPT AES_ENCRYPT && ANY_VALUE ASCII ASIN ATAN ATAN2 ATAN BENCHMARK BIN_TO_UUID BIT_AND BIT_COUNT BIT_LENGTH BIT_OR BIT_XOR CAN_ACCESS_COLUMN CAN_ACCESS_DATABASE CAN_ACCESS_TABLE CAN_ACCESS_USER CAN_ACCESS_VIEW CEIL CEILING CHAR_LENGTH CHARACTER_LENGTH COERCIBILITY CONCAT_WS CONNECTION_ID CONVERT_TZ COT COUNT COUNTDISTINCT CRC32 CURDATE CURRENT_ROLE CURTIME DATE_ADD DATE_FORMAT DATE_SUB DATEDIFF DAY DAYNAME DAYOFMONTH DAYOFWEEK DAYOFYEAR DEGREES ELT EXPORT_SET ExtractValue FIND_IN_SET FLOOR FORMAT_BYTES FORMAT_PICO_TIME FROM_BASE64 FROM_DAYS FROM_UNIXTIME GeomCollection GeometryCollection GET_DD_COLUMN_PRIVILEGES GET_DD_CREATE_OPTIONS GET_DD_INDEX_SUB_PART_LENGTH GET_FORMAT GET_LOCK GREATEST GTID_SUBSET GTID_SUBTRACT HOUR ICU_VERSION IFNULL INET_ATON INET_NTOA INET6_ATON INET6_NTOA INSTR INTERNAL_AUTO_INCREMENT INTERNAL_AVG_ROW_LENGTH INTERNAL_CHECK_TIME INTERNAL_CHECKSUM INTERNAL_DATA_FREE INTERNAL_DATA_LENGTH INTERNAL_DD_CHAR_LENGTH INTERNAL_GET_COMMENT_OR_ERROR INTERNAL_GET_ENABLED_ROLE_JSON INTERNAL_GET_HOSTNAME INTERNAL_GET_USERNAME INTERNAL_GET_VIEW_WARNING_OR_ERROR INTERNAL_INDEX_COLUMN_CARDINALITY INTERNAL_INDEX_LENGTH INTERNAL_IS_ENABLED_ROLE INTERNAL_IS_MANDATORY_ROLE INTERNAL_KEYS_DISABLED INTERNAL_MAX_DATA_LENGTH INTERNAL_TABLE_ROWS INTERNAL_UPDATE_TIME IS_FREE_LOCK IS_IPV4 IS_IPV4_COMPAT IS_IPV4_MAPPED IS_IPV6 IS_USED_LOCK IS_UUID ISNULL JSON_ARRAY JSON_ARRAY_APPEND JSON_ARRAY_INSERT JSON_ARRAYAGG JSON_CONTAINS JSON_CONTAINS_PATH JSON_DEPTH JSON_INSERT JSON_KEYS JSON_LENGTH JSON_MERGE JSON_MERGE_PATCH JSON_MERGE_PRESERVE JSON_OBJECT JSON_OBJECTAGG JSON_OVERLAPS JSON_PRETTY JSON_QUOTE JSON_REMOVE JSON_REPLACE JSON_SCHEMA_VALID JSON_SCHEMA_VALIDATION_REPORT JSON_SEARCH JSON_SET JSON_STORAGE_FREE JSON_STORAGE_SIZE JSON_TYPE JSON_VALID JSON_VALUE LAST_DAY LCASE LEAST LineString LOAD_FILE LOCATE LOG10 LOG2 LPAD LTRIM MAKE_SET MAKEDATE MAKETIME MASTER_POS_WAIT MBRContains MBRCoveredBy MBRCovers MBRDisjoint MBREquals MBRIntersects MBROverlaps MBRTouches MBRWithin MD5 MICROSECOND MID MINUTE MONTH MONTHNAME MultiLineString MultiPoint MultiPolygon NAME_CONST NULLIF OCT OCTET_LENGTH || PERIOD_ADD PERIOD_DIFF Point Polygon POSITION POW POWER PS_CURRENT_THREAD_ID PS_THREAD_ID QUARTER RADIANS RANDOM_BYTES REGEXP_INSTR REGEXP_LIKE REGEXP_REPLACE REGEXP_SUBSTR RELEASE_ALL_LOCKS RELEASE_LOCK REVERSE RLIKE ROLES_GRAPHML ROUND ROW_COUNT RPAD RTRIM SEC_TO_TIME SESSION_USER SHA1 SHA2 SLEEP SOUNDEX SOUNDS SOURCE_POS_WAIT SQRT ST_Area ST_AsBinary ST_AsWKB ST_AsGeoJSON ST_AsText ST_AsWKT ST_Buffer ST_Buffer_Strategy ST_Centroid ST_Collect ST_Contains ST_ConvexHull ST_Crosses ST_Difference ST_Dimension ST_Disjoint ST_Distance ST_Distance_Sphere ST_EndPoint ST_Envelope ST_Equals ST_ExteriorRing ST_FrechetDistance ST_GeoHash ST_GeomCollFromText ST_GeometryCollectionFromText ST_GeomCollFromTxt ST_GeomCollFromWKB ST_GeometryCollectionFromWKB ST_GeometryN ST_GeometryType ST_GeomFromGeoJSON ST_GeomFromText ST_GeometryFromText ST_GeomFromWKB ST_GeometryFromWKB ST_HausdorffDistance ST_InteriorRingN ST_Intersection ST_Intersects ST_IsClosed ST_IsEmpty ST_IsSimple ST_IsValid ST_LatFromGeoHash ST_Latitude ST_Length ST_LineFromText ST_LineStringFromText ST_LineFromWKB ST_LineStringFromWKB ST_LineInterpolatePoint ST_LineInterpolatePoints ST_LongFromGeoHash ST_Longitude ST_MakeEnvelope ST_MLineFromText ST_MultiLineStringFromText ST_MLineFromWKB ST_MultiLineStringFromWKB ST_MPointFromText ST_MultiPointFromText ST_MPointFromWKB ST_MultiPointFromWKB ST_MPolyFromText ST_MultiPolygonFromText ST_MPolyFromWKB ST_MultiPolygonFromWKB ST_NumGeometries ST_NumInteriorRing ST_NumInteriorRings ST_NumPoints ST_Overlaps ST_PointAtDistance ST_PointFromGeoHash ST_PointFromText ST_PointFromWKB ST_PointN ST_PolyFromText ST_PolygonFromText ST_PolyFromWKB ST_PolygonFromWKB ST_Simplify ST_SRID ST_StartPoint ST_SwapXY ST_SymDifference ST_Touches ST_Transform ST_Union ST_Validate ST_Within ST_X ST_Y STATEMENT_DIGEST STATEMENT_DIGEST_TEXT STDDEV STDDEV_POP STDDEV_SAMP STR_TO_DATE STRCMP SUBDATE SUBSTRING_INDEX SUBTIME SYSDATE SYSTEM_USER TAN TIME_FORMAT TIME_TO_SEC TIMEDIFF TO_BASE64 TO_DAYS TO_SECONDS TRIM UCASE UNCOMPRESS UNCOMPRESSED_LENGTH UNHEX UNIX_TIMESTAMP UpdateXML UPPER UUID_SHORT UUID_TO_BIN VALIDATE_PASSWORD_STRENGTH VAR_POP VAR_SAMP VARIANCE VERSION WAIT_FOR_EXECUTED_GTID_SET WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS WAIT_FOR_EXECUTED_GTID_SET WEEK WEEKDAY WEEKOFYEAR WEIGHT_STRING YEARWEEK ```

EDIT: There might be even more because I saw a section in the yacc parser config which said MySQL reserved words that are unused by this grammar will map to this token. As these also aren't used by the grammar

GuptaManan100 commented 3 years ago

@ritwizsinha Yes that is a great first step in gap analysis!

ritwizsinha commented 3 years ago

@GuptaManan100 are you available on slack we can discuss there

GuptaManan100 commented 3 years ago

Yes I am available on VItess slack

aribalam commented 3 years ago

@GuptaManan100 Looking at the sql.y file, there are already support for parsing the COLLATE and CHARACTER SET constructs. The same is also present in the parse_test.go. Could you please confirm?

GuptaManan100 commented 3 years ago

@aribalam please take a look at this thread in Vitess slack https://vitess.slack.com/archives/C0PQY0PTK/p1628839955078000

Thirumalai-Shaktivel commented 3 years ago

I would recommend keeping track of all the Built-In Functions and Operators in MySQL 5.7 which has already been implemented.

Marked one is already parsed by the sqlparser, Rest has to be implemented

[x] &
Bitwise AND
[x] >
Greater than operator
[ ] >>
Right shift
[ ] >=
Greater than or equal operator
[x] <
Less than operator
[ ] <>, !=
Not equal operator
[ ] <<
Left shift
[ ] <=
Less than or equal operator
[ ] <=>
NULL-safe equal to operator
[x] %, MOD
Modulo operator
[x] *
Multiplication operator
[x] +
Addition operator
[x] -
Minus operator
[ ] -
Change the sign of the argument
[ ] ->
Return value from JSON column after evaluating path; equivalent to JSON_EXTRACT().
[ ] ->>
Return value from JSON column after evaluating path and unquoting the result; equivalent to JSON_UNQUOTE(JSON_EXTRACT()).
[ ] /
Division operator
[ ] :=
Assign a value
[ ] =
Assign a value (as part of a SET statement, or as part of the SET clause in an UPDATE statement)
[ ] =
Equal operator
[x] ^
Bitwise XOR
[ ] ABS()
Return the absolute value
[ ] ACOS()
Return the arc cosine
[ ] ADDDATE()
Add time values (intervals) to a date value
[ ] ADDTIME()
Add time
[ ] AES_DECRYPT()
Decrypt using AES
[ ] AES_ENCRYPT()
Encrypt using AES
[x] AND, &&
Logical AND
[ ] ANY_VALUE()
Suppress ONLY_FULL_GROUP_BY value rejection
[ ] Area()
Return Polygon or MultiPolygon area
[ ] AsBinary(), AsWKB()
Convert from internal geometry format to WKB
[ ] ASCII()
Return numeric value of left-most character
[ ] ASIN()
Return the arc sine
[ ] AsText(), AsWKT()
Convert from internal geometry format to WKT
[ ] ATAN()
Return the arc tangent
[ ] ATAN2(), ATAN()
Return the arc tangent of the two arguments
[ ] AVG()
Return the average value of the argument
[ ] BENCHMARK()
Repeatedly execute an expression
[ ] BETWEEN ... AND ...
Whether a value is within a range of values
[ ] BIN()
Return a string containing binary representation of a number
[ ] BINARY
Cast a string to a binary string
[ ] BIT_AND()
Return bitwise AND
[ ] BIT_COUNT()
Return the number of bits that are set
[ ] BIT_LENGTH()
Return length of argument in bits
[ ] BIT_OR()
Return bitwise OR
[ ] BIT_XOR()
Return bitwise XOR
[ ] Buffer()
Return geometry of points within given distance from geometry
[x] CASE
Case operator
[x] CAST()
Cast a value as a certain type
[ ] CEIL()
Return the smallest integer value not less than the argument
[ ] CEILING()
Return the smallest integer value not less than the argument
[ ] Centroid()
Return centroid as a point
[ ] CHAR()
Return the character for each integer passed
[ ] CHAR_LENGTH()
Return number of characters in argument
[ ] CHARACTER_LENGTH()
Synonym for CHAR_LENGTH()
[ ] CHARSET()
Return the character set of the argument
[ ] COALESCE()
Return the first non-NULL argument
[ ] COERCIBILITY()
Return the collation coercibility value of the string argument
[ ] COLLATION()
Return the collation of the string argument
[ ] COMPRESS()
Return result as a binary string
[ ] CONCAT()
Return concatenated string
[ ] CONCAT_WS()
Return concatenate with separator
[ ] CONNECTION_ID()
Return the connection ID (thread ID) for the connection
[ ] Contains()
Whether MBR of one geometry contains MBR of another
[ ] CONV()
Convert numbers between different number bases
[x] CONVERT()
Cast a value as a certain type
[ ] CONVERT_TZ()
Convert from one time zone to another
[ ] ConvexHull()
Return convex hull of geometry
[ ] COS()
Return the cosine
[ ] COT()
Return the cotangent
[ ] COUNT()
Return a count of the number of rows returned
[ ] COUNT(DISTINCT)
Return the count of a number of different values
[ ] CRC32()
Compute a cyclic redundancy check value
[ ] Crosses()
Whether one geometry crosses another
[ ] CURDATE()
Return the current date
[x] CURRENT_DATE(), CURRENT_DATE
Synonyms for CURDATE()
[x] CURRENT_TIME(), CURRENT_TIME
Synonyms for CURTIME()
[x] CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP
Synonyms for NOW()
[x] CURRENT_USER(), CURRENT_USER
The authenticated user name and host name
[ ] CURTIME()
Return the current time
[x] DATABASE()
Return the default (current) database name
[ ] DATE()
Extract the date part of a date or datetime expression
[ ] DATE_ADD()
Add time values (intervals) to a date value
[ ] DATE_FORMAT()
Format date as specified
[ ] DATE_SUB()
Subtract a time value (interval) from a date
[ ] DATEDIFF()
Subtract two dates
[ ] DAY()
Synonym for DAYOFMONTH()
[ ] DAYNAME()
Return the name of the weekday
[ ] DAYOFMONTH()
Return the day of the month (0-31)
[ ] DAYOFWEEK()
Return the weekday index of the argument
[ ] DAYOFYEAR()
Return the day of the year (1-366)
[ ] DECODE()
Decode a string encrypted using ENCODE()
[ ] DEFAULT()
Return the default value for a table column
[ ] DEGREES()
Convert radians to degrees
[ ] DES_DECRYPT()
Decrypt a string
[ ] DES_ENCRYPT()
Encrypt a string
[ ] Dimension()
Dimension of geometry
[ ] Disjoint()
Whether MBRs of two geometries are disjoint
[ ] Distance()
The distance of one geometry from another
[ ] DIV
Integer division
[ ] ELT()
Return string at index number
[ ] ENCODE()
Encode a string
[ ] ENCRYPT()
Encrypt a string
[ ] EndPoint()
End Point of LineString
[ ] Envelope()
Return MBR of geometry
[ ] Equals()
Whether MBRs of two geometries are equal
[ ] EXP()
Raise to the power of
[ ] EXPORT_SET()
Return a string such that for every bit set in the value bits, you get an on string and for every unset bit, you get an off string
[ ] ExteriorRing()
Return exterior ring of Polygon
[ ] EXTRACT()
Extract part of a date
[ ] ExtractValue()
Extract a value from an XML string using XPath notation
[ ] FIELD()
Index (position) of first argument in subsequent arguments
[ ] FIND_IN_SET()
Index (position) of first argument within second argument
[ ] FLOOR()
Return the largest integer value not greater than the argument
[ ] FORMAT()
Return a number formatted to specified number of decimal places
[ ] FOUND_ROWS()
For a SELECT with a LIMIT clause, the number of rows that would be returned were there no LIMIT clause
[ ] FROM_BASE64()
Decode base64 encoded string and return result
[ ] FROM_DAYS()
Convert a day number to a date
[ ] FROM_UNIXTIME()
Format Unix timestamp as a date
[ ] GeomCollFromText(), GeometryCollectionFromText()
Return geometry collection from WKT
[ ] GeomCollFromWKB(), GeometryCollectionFromWKB()
Return geometry collection from WKB
[ ] GeometryCollection()
Construct geometry collection from geometries
[ ] GeometryN()
Return N-th geometry from geometry collection
[ ] GeometryType()
Return name of geometry type
[ ] GeomFromText(), GeometryFromText()
Return geometry from WKT
[ ] GeomFromWKB(), GeometryFromWKB()
Return geometry from WKB
[ ] GET_FORMAT()
Return a date format string
[ ] GET_LOCK()
Get a named lock
[ ] GLength()
Return length of LineString
[ ] GREATEST()
Return the largest argument
[x] GROUP_CONCAT()
Return a concatenated string
[ ] GTID_SUBSET()
Return true if all GTIDs in subset are also in set; otherwise false.
[ ] GTID_SUBTRACT()
Return all GTIDs in set that are not in subset.
[ ] HEX()
Hexadecimal representation of decimal or string value
[ ] HOUR()
Extract the hour
[x] IF()
If/else construct
[ ] IFNULL()
Null if/else construct
[ ] IN()
Whether a value is within a set of values
[ ] INET_ATON()
Return the numeric value of an IP address
[ ] INET_NTOA()
Return the IP address from a numeric value
[ ] INET6_ATON()
Return the numeric value of an IPv6 address
[ ] INET6_NTOA()
Return the IPv6 address from a numeric value
[ ] INSERT()
Insert substring at specified position up to specified number of characters
[ ] INSTR()
Return the index of the first occurrence of substring
[ ] InteriorRingN()
Return N-th interior ring of Polygon
[ ] Intersects()
Whether MBRs of two geometries intersect
[ ] INTERVAL()
Return the index of the argument that is less than the first argument
[ ] IS
Test a value against a boolean
[ ] IS_FREE_LOCK()
Whether the named lock is free
[ ] IS_IPV4()
Whether argument is an IPv4 address
[ ] IS_IPV4_COMPAT()
Whether argument is an IPv4-compatible address
[ ] IS_IPV4_MAPPED()
Whether argument is an IPv4-mapped address
[ ] IS_IPV6()
Whether argument is an IPv6 address
[ ] IS NOT
Test a value against a boolean
[ ] IS NOT NULL
NOT NULL value test
[ ] IS NULL
NULL value test
[ ] IS_USED_LOCK()
Whether the named lock is in use; return connection identifier if true
[ ] IsClosed()
Whether a geometry is closed and simple
[ ] IsEmpty()
Whether a geometry is empty
[ ] ISNULL()
Test whether the argument is NULL
[ ] IsSimple()
Whether a geometry is simple
[ ] JSON_APPEND()
Append data to JSON document
[ ] JSON_ARRAY()
Create JSON array
[ ] JSON_ARRAY_APPEND()
Append data to JSON document
[ ] JSON_ARRAY_INSERT()
Insert into JSON array
[ ] JSON_ARRAYAGG()
Return result set as a single JSON array`
[ ] JSON_CONTAINS()
Whether JSON document contains specific object at path
[ ] JSON_CONTAINS_PATH()
Whether JSON document contains any data at path
[ ] JSON_DEPTH()
Maximum depth of JSON document
[ ] JSON_EXTRACT()
Return data from JSON document
[ ] JSON_INSERT()
Insert data into JSON document
[ ] JSON_KEYS()
Array of keys from JSON document
[ ] JSON_LENGTH()
Number of elements in JSON document
[ ] JSON_MERGE()
Merge JSON documents, preserving duplicate keys. Deprecated synonym for JSON_MERGE_PRESERVE()
[ ] JSON_MERGE_PATCH()
Merge JSON documents, replacing values of duplicate keys`
[ ] JSON_MERGE_PRESERVE()
Merge JSON documents, preserving duplicate keys`
[ ] JSON_OBJECT()
Create JSON object
[ ] JSON_OBJECTAGG()
Return result set as a single JSON object`
[ ] JSON_PRETTY()
Print a JSON document in human-readable format`
[ ] JSON_QUOTE()
Quote JSON document
[ ] JSON_REMOVE()
Remove data from JSON document
[ ] JSON_REPLACE()
Replace values in JSON document
[ ] JSON_SEARCH()
Path to value within JSON document
[ ] JSON_SET()
Insert data into JSON document
[ ] JSON_STORAGE_SIZE()
Space used for storage of binary representation of a JSON document`
[ ] JSON_TYPE()
Type of JSON value
[ ] JSON_UNQUOTE()
Unquote JSON value
[ ] JSON_VALID()
Whether JSON value is valid
[ ] LAST_DAY
Return the last day of the month for the argument
[ ] LAST_INSERT_ID()
Value of the AUTOINCREMENT column for the last INSERT
[ ] LCASE()
Synonym for LOWER()
[ ] LEAST()
Return the smallest argument
[x] LEFT()
Return the leftmost number of characters as specified
[ ] LENGTH()
Return the length of a string in bytes
[ ] LIKE
Simple pattern matching
[ ] LineFromText(), LineStringFromText()
Construct LineString from WKT
[ ] LineFromWKB(), LineStringFromWKB()
Construct LineString from WKB
[ ] LineString()
Construct LineString from Point values
[ ] LN()
Return the natural logarithm of the argument
[ ] LOAD_FILE()
Load the named file
[x] LOCALTIME(), LOCALTIME
Synonym for NOW()
[x] LOCALTIMESTAMP, LOCALTIMESTAMP()
Synonym for NOW()
[ ] LOCATE()
Return the position of the first occurrence of substring
[ ] LOG()
Return the natural logarithm of the first argument
[ ] LOG10()
Return the base-10 logarithm of the argument
[ ] LOG2()
Return the base-2 logarithm of the argument
[ ] LOWER()
Return the argument in lowercase
[ ] LPAD()
Return the string argument, left-padded with the specified string
[ ] LTRIM()
Remove leading spaces
[ ] MAKE_SET()
Return a set of comma-separated strings that have the corresponding bit in bits set
[ ] MAKEDATE()
Create a date from the year and day of year
[ ] MAKETIME()
Create time from hour, minute, second
[ ] MASTER_POS_WAIT()
Block until the replica has read and applied all updates up to the specified position
[ ] MATCH
Perform full-text search
[ ] MAX()
Return the maximum value
[ ] MBRContains()
Whether MBR of one geometry contains MBR of another
[ ] MBRCoveredBy()
Whether one MBR is covered by another
[ ] MBRCovers()
Whether one MBR covers another
[ ] MBRDisjoint()
Whether MBRs of two geometries are disjoint
[ ] MBREqual()
Whether MBRs of two geometries are equal
[ ] MBREquals()
Whether MBRs of two geometries are equal
[ ] MBRIntersects()
Whether MBRs of two geometries intersect
[ ] MBROverlaps()
Whether MBRs of two geometries overlap
[ ] MBRTouches()
Whether MBRs of two geometries touch
[ ] MBRWithin()
Whether MBR of one geometry is within MBR of another
[ ] MD5()
Calculate MD5 checksum
[ ] MICROSECOND()
Return the microseconds from argument
[ ] MID()
Return a substring starting from the specified position
[ ] MIN()
Return the minimum value
[ ] MINUTE()
Return the minute from the argument
[ ] MLineFromText(), MultiLineStringFromText()
Construct MultiLineString from WKT
[ ] MLineFromWKB(), MultiLineStringFromWKB()
Construct MultiLineString from WKB
[x] MOD()
Return the remainder
[ ] MONTH()
Return the month from the date passed
[ ] MONTHNAME()
Return the name of the month
[ ] MPointFromText(), MultiPointFromText()
Construct MultiPoint from WKT
[ ] MPointFromWKB(), MultiPointFromWKB()
Construct MultiPoint from WKB
[ ] MPolyFromText(), MultiPolygonFromText()
Construct MultiPolygon from WKT
[ ] MPolyFromWKB(), MultiPolygonFromWKB()
Construct MultiPolygon from WKB
[ ] MultiLineString()
Contruct MultiLineString from LineString values
[ ] MultiPoint()
Construct MultiPoint from Point values
[ ] MultiPolygon()
Construct MultiPolygon from Polygon values
[ ] NAME_CONST()
Cause the column to have the given name
[ ] NOT, !
Negates value
[ ] NOT BETWEEN ... AND ...
Whether a value is not within a range of values
[ ] NOT IN()
Whether a value is not within a set of values
[ ] NOT LIKE
Negation of simple pattern matching
[ ] NOT REGEXP
Negation of REGEXP
[ ] NOW()
Return the current date and time
[ ] NULLIF()
Return NULL if expr1 = expr2
[ ] NumGeometries()
Return number of geometries in geometry collection
[ ] NumInteriorRings()
Return number of interior rings in Polygon
[ ] NumPoints()
Return number of points in LineString
[ ] OCT()
Return a string containing octal representation of a number
[ ] OCTET_LENGTH()
Synonym for LENGTH()
[ ] OR, ||
Logical OR
[ ] ORD()
Return character code for leftmost character of the argument
[ ] Overlaps()
Whether MBRs of two geometries overlap
[ ] PASSWORD()
Calculate and return a password string
[ ] PERIOD_ADD()
Add a period to a year-month
[ ] PERIOD_DIFF()
Return the number of months between periods
[ ] PI()
Return the value of pi
[ ] Point()
Construct Point from coordinates
[ ] PointFromText()
Construct Point from WKT
[ ] PointFromWKB()
Construct Point from WKB
[ ] PointN()
Return N-th point from LineString
[ ] PolyFromText(), PolygonFromText()
Construct Polygon from WKT
[ ] PolyFromWKB(), PolygonFromWKB()
Construct Polygon from WKB
[ ] Polygon()
Construct Polygon from LineString arguments
[ ] POSITION()
Synonym for LOCATE()
[ ] POW()
Return the argument raised to the specified power
[ ] POWER()
Return the argument raised to the specified power
[ ] PROCEDURE ANALYSE()
Analyze the results of a query
[ ] QUARTER()
Return the quarter from a date argument
[ ] QUOTE()
Escape the argument for use in an SQL statement
[ ] RADIANS()
Return argument converted to radians
[ ] RAND()
Return a random floating-point value
[ ] RANDOM_BYTES()
Return a random byte vector
[ ] REGEXP
Whether string matches regular expression
[ ] RELEASE_ALL_LOCKS()
Release all current named locks
[ ] RELEASE_LOCK()
Release the named lock
[ ] REPEAT()
Repeat a string the specified number of times
[x] REPLACE()
Replace occurrences of a specified string
[ ] REVERSE()
Reverse the characters in a string
[x] RIGHT()
Return the specified rightmost number of characters
[ ] RLIKE
Whether string matches regular expression
[ ] ROUND()
Round the argument
[ ] ROW_COUNT()
The number of rows updated
[ ] RPAD()
Append string the specified number of times
[ ] RTRIM()
Remove trailing spaces
[x] SCHEMA()
Synonym for DATABASE()
[ ] SEC_TO_TIME()
Converts seconds to 'hh:mm:ss' format
[ ] SECOND()
Return the second (0-59)
[ ] SESSION_USER()
Synonym for USER()
[ ] SHA1(), SHA()
Calculate an SHA-1 160-bit checksum
[ ] SHA2()
Calculate an SHA-2 checksum
[ ] SIGN()
Return the sign of the argument
[ ] SIN()
Return the sine of the argument
[ ] SLEEP()
Sleep for a number of seconds
[ ] SOUNDEX()
Return a soundex string
[ ] SOUNDS LIKE
Compare sounds
[ ] SPACE()
Return a string of the specified number of spaces
[ ] SQRT()
Return the square root of the argument
[ ] SRID()
Return spatial reference system ID for geometry
[ ] ST_Area()
Return Polygon or MultiPolygon area
[ ] ST_AsBinary(), ST_AsWKB()
Convert from internal geometry format to WKB
[ ] ST_AsGeoJSON()
Generate GeoJSON object from geometry
[ ] ST_AsText(), ST_AsWKT()
Convert from internal geometry format to WKT
[ ] ST_Buffer()
Return geometry of points within given distance from geometry
[ ] ST_Buffer_Strategy()
Produce strategy option for ST_Buffer()
[ ] ST_Centroid()
Return centroid as a point
[ ] ST_Contains()
Whether one geometry contains another
[ ] ST_ConvexHull()
Return convex hull of geometry
[ ] ST_Crosses()
Whether one geometry crosses another
[ ] ST_Difference()
Return point set difference of two geometries
[ ] ST_Dimension()
Dimension of geometry
[ ] ST_Disjoint()
Whether one geometry is disjoint from another
[ ] ST_Distance()
The distance of one geometry from another
[ ] ST_Distance_Sphere()
Minimum distance on earth between two geometries
[ ] ST_EndPoint()
End Point of LineString
[ ] ST_Envelope()
Return MBR of geometry
[ ] ST_Equals()
Whether one geometry is equal to another
[ ] ST_ExteriorRing()
Return exterior ring of Polygon
[ ] ST_GeoHash()
Produce a geohash value
[ ] ST_GeomCollFromText(), ST_GeometryCollectionFromText(), ST_GeomCollFromTxt()
Return geometry collection from WKT
[ ] ST_GeomCollFromWKB(), ST_GeometryCollectionFromWKB()
Return geometry collection from WKB
[ ] ST_GeometryN()
Return N-th geometry from geometry collection
[ ] ST_GeometryType()
Return name of geometry type
[ ] ST_GeomFromGeoJSON()
Generate geometry from GeoJSON object
[ ] ST_GeomFromText(), ST_GeometryFromText()
Return geometry from WKT
[ ] ST_GeomFromWKB(), ST_GeometryFromWKB()
Return geometry from WKB
[ ] ST_InteriorRingN()
Return N-th interior ring of Polygon
[ ] ST_Intersection()
Return point set intersection of two geometries
[ ] ST_Intersects()
Whether one geometry intersects another
[ ] ST_IsClosed()
Whether a geometry is closed and simple
[ ] ST_IsEmpty()
Whether a geometry is empty
[ ] ST_IsSimple()
Whether a geometry is simple
[ ] ST_IsValid()
Whether a geometry is valid
[ ] ST_LatFromGeoHash()
Return latitude from geohash value
[ ] ST_Length()
Return length of LineString
[ ] ST_LineFromText(), ST_LineStringFromText()
Construct LineString from WKT
[ ] ST_LineFromWKB(), ST_LineStringFromWKB()
Construct LineString from WKB
[ ] ST_LongFromGeoHash()
Return longitude from geohash value
[ ] ST_MakeEnvelope()
Rectangle around two points
[ ] ST_MLineFromText(), ST_MultiLineStringFromText()
Construct MultiLineString from WKT
[ ] ST_MLineFromWKB(), ST_MultiLineStringFromWKB()
Construct MultiLineString from WKB
[ ] ST_MPointFromText(), ST_MultiPointFromText()
Construct MultiPoint from WKT
[ ] ST_MPointFromWKB(), ST_MultiPointFromWKB()
Construct MultiPoint from WKB
[ ] ST_MPolyFromText(), ST_MultiPolygonFromText()
Construct MultiPolygon from WKT
[ ] ST_MPolyFromWKB(), ST_MultiPolygonFromWKB()
Construct MultiPolygon from WKB
[ ] ST_NumGeometries()
Return number of geometries in geometry collection
[ ] ST_NumInteriorRing(), ST_NumInteriorRings()
Return number of interior rings in Polygon
[ ] ST_NumPoints()
Return number of points in LineString
[ ] ST_Overlaps()
Whether one geometry overlaps another
[ ] ST_PointFromGeoHash()
Convert geohash value to POINT value
[ ] ST_PointFromText()
Construct Point from WKT
[ ] ST_PointFromWKB()
Construct Point from WKB
[ ] ST_PointN()
Return N-th point from LineString
[ ] ST_PolyFromText(), ST_PolygonFromText()
Construct Polygon from WKT
[ ] ST_PolyFromWKB(), ST_PolygonFromWKB()
Construct Polygon from WKB
[ ] ST_Simplify()
Return simplified geometry
[ ] ST_SRID()
Return spatial reference system ID for geometry
[ ] ST_StartPoint()
Start Point of LineString
[ ] ST_SymDifference()
Return point set symmetric difference of two geometries
[ ] ST_Touches()
Whether one geometry touches another
[ ] ST_Union()
Return point set union of two geometries
[ ] ST_Validate()
Return validated geometry
[ ] ST_Within()
Whether one geometry is within another
[ ] ST_X()
Return X coordinate of Point
[ ] ST_Y()
Return Y coordinate of Point
[ ] StartPoint()
Start Point of LineString
[ ] STD()
Return the population standard deviation
[ ] STDDEV()
Return the population standard deviation
[ ] STDDEV_POP()
Return the population standard deviation
[ ] STDDEV_SAMP()
Return the sample standard deviation
[ ] STR_TO_DATE()
Convert a string to a date
[ ] STRCMP()
Compare two strings
[ ] SUBDATE()
Synonym for DATE_SUB() when invoked with three arguments
[x] SUBSTR()
Return the substring as specified
[x] SUBSTRING()
Return the substring as specified
[ ] SUBSTRING_INDEX()
Return a substring from a string before the specified number of occurrences of the delimiter
[ ] SUBTIME()
Subtract times
[ ] SUM()
Return the sum
[ ] SYSDATE()
Return the time at which the function executes
[ ] SYSTEM_USER()
Synonym for USER()
[ ] TAN()
Return the tangent of the argument
[ ] TIME()
Extract the time portion of the expression passed
[ ] TIME_FORMAT()
Format as time
[ ] TIME_TO_SEC()
Return the argument converted to seconds
[ ] TIMEDIFF()
Subtract time
[ ] TIMESTAMP()
With a single argument, this function returns the date or datetime expression; with two arguments, the sum of the arguments
[x] TIMESTAMPADD()
Add an interval to a datetime expression
[x] TIMESTAMPDIFF()
Subtract an interval from a datetime expression
[ ] TO_BASE64()
Return the argument converted to a base-64 string
[ ] TO_DAYS()
Return the date argument converted to days
[ ] TO_SECONDS()
Return the date or datetime argument converted to seconds since Year 0
[ ] Touches()
Whether one geometry touches another
[ ] TRIM()
Remove leading and trailing spaces
[ ] TRUNCATE()
Truncate to specified number of decimal places
[ ] UCASE()
Synonym for UPPER()
[ ] UNCOMPRESS()
Uncompress a string compressed
[ ] UNCOMPRESSED_LENGTH()
Return the length of a string before compression
[ ] UNHEX()
Return a string containing hex representation of a number
[ ] UNIX_TIMESTAMP()
Return a Unix timestamp
[ ] UpdateXML()
Return replaced XML fragment
[ ] UPPER()
Convert to uppercase
[ ] USER()
The user name and host name provided by the client
[x] UTC_DATE()
Return the current UTC date
[x] UTC_TIME()
Return the current UTC time
[x] UTC_TIMESTAMP()
Return the current UTC date and time
[ ] UUID()
Return a Universal Unique Identifier (UUID)
[ ] UUID_SHORT()
Return an integer-valued universal identifier
[ ] VALIDATE_PASSWORD_STRENGTH()
Determine strength of password
[x] VALUES()
Define the values to be used during an INSERT
[ ] VAR_POP()
Return the population standard variance
[ ] VAR_SAMP()
Return the sample variance
[ ] VARIANCE()
Return the population standard variance
[ ] VERSION()
Return a string that indicates the MySQL server version
[ ] WAIT_FOR_EXECUTED_GTID_SET()
Wait until the given GTIDs have executed on the replica.
[ ] WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS()
Use WAIT_FOR_EXECUTED_GTID_SET().
[ ] WEEK()
Return the week number
[ ] WEEKDAY()
Return the weekday index
[ ] WEEKOFYEAR()
Return the calendar week of the date (1-53)
[ ] WEIGHT_STRING()
Return the weight string for a string
[ ] Within()
Whether MBR of one geometry is within MBR of another
[ ] X()
Return X coordinate of Point
[ ] XOR
Logical XOR
[ ] Y()
Return Y coordinate of Point
[ ] YEAR()
Return the year
[ ] YEARWEEK()
Return the year and week
[x] |
Bitwise OR
[x] ~
Bitwise inversion

GuptaManan100 commented 3 years ago

That is good work documenting the functions that we do not handle explicitly in MySQL @Thirumalai-Shaktivel.

tharun208 commented 2 years ago

@GuptaManan100 I like to work on this as of the LFX program. I am interested in databases and I would be familiarising myself with Vitess and getting to know better about Vitess. Any help is appreciated.

K-Kumar-01 commented 2 years ago

@GuptaManan100 I would like to work apply for this project under LFX. I am not familiar with Vitess as of now so currently I am going through the codebase and familiarizing myself with the concepts. I do have some work flow which I have thought of to follow and I would like to discuss it with you, if possible.

GuptaManan100 commented 2 years ago

All discussions about the project will happen on Slack. Here is the link to the general channel - https://vitess.slack.com/archives/C0PQY0PTK

GuptaManan100 commented 2 years ago

Added #9682 to the description for tracking progress

Weijun-H commented 2 years ago

Hi @GuptaManan100 , I am really interested in this project under LFX. Could you give me some suggestions how to start it? Should I figure out which functions is still missing?

GuptaManan100 commented 2 years ago

@Weijun-H, the issue has the list of things that the parser is missing. Anything that is unticked, can be worked on. You can look at any of the linked PRs to see where the tests reside and how to make parser changes. You can also refer to https://vitess.io/docs/15.0/contributing/contributing-to-ast-parser/ and https://vitess.io/docs/15.0/contributing/sample-first-issue/ for guidance.

monalisha31 commented 2 years ago

Hi @GuptaManan100, I would like apply for this project under the LFX mentorship program. This project seems interesting. Looking forward to contributing to it.

skant7 commented 2 years ago

Hi @GuptaManan100 ,I'm interested in working on this as part of the LFX Mentorship program. Can I start working on the String functions just to get a better understanding of the project or should I focus on the spatial functions to start with ?

GuptaManan100 commented 2 years ago

@skant7 Sure, go right ahead, pick up whichever one you feel most comfortable with.

SAEb-ai commented 2 years ago

Hi @GuptaManan100 !! I am interested to contribute to this project under the LFX mentorship program and hence I applied to it now. Looking forward to contribute. Thanks!!

GuptaManan100 commented 2 years ago

Everyone interested in the LFX project, please join the Vitess slack, #lfx-winter-2022 channel

ktwillcode commented 1 year ago

hi @GuptaManan100 Kartikeya this side, I applied for the mentorship and submitted a Cover letter and Resume. Is there anything more to complete the process?

GuptaManan100 commented 1 year ago

@ktwillcode There is nothing else required to complete the process. You can try your hand at implementing one of the functions that Vitess doesn’t already have parsing for. It will give you an idea on how the project work will be and it will give us confidence in your ability to do the project.

ktwillcode commented 1 year ago

@ktwillcode There is nothing else required to complete the process. You can try your hand at implementing one of the functions that Vitess doesn’t already have parsing for. It will give you an idea on how the project work will be and it will give us confidence in your ability to do the project.

@GuptaManan100 Okay. Looking forward to work on it

Ayman161803 commented 1 year ago

The following type of functions are yet to be implemented

vitessio / vitess

Track complete parsing support in Vitess #8604

Description

8797

8821

8884

8918

9006

9011

9029

9075

8691