reata / sqllineage

SQL Lineage Analysis Tool powered by Python
MIT License
1.32k stars 240 forks source link

Parse stored procedures using sqllineage #643

Open godhao123 opened 2 months ago

godhao123 commented 2 months ago

Really love sqllineage! I have a question: I want to parse SQL statements from stored procedures. Is there a good method to extract valid SQL statements from stored procedures?

kendn1993 commented 2 months ago

me too

maoxingda commented 2 months ago

Preprocess the contents of the stored procedure before using SQLLineage to analyze the lineage.

godhao123 commented 2 months ago

Preprocess the contents of the stored procedure before using SQLLineage to analyze the lineage.

Could you please advise on how to effectively preprocess stored procedures, such as removing irrelevant syntax keywords and extracting complete, valid SQL? I am currently using regular expression matching to preprocess stored procedures, but this method cannot guarantee completeness and accuracy

maoxingda commented 2 months ago
godhao123 commented 2 months ago
  • My own approach is as follows:

    • Standardize the writing of stored procedures (limit certain writing methods, such as supporting three levels of nested judgment statements, etc.)
    • Use regular expressions to extract grammatically correct SQL, and then use sqllineage to parse the lineage

Thank you very much for your response. I will strive to do better.

kendn1993 commented 2 months ago

I use sqlparse to parse procedures.