search

opensearch-project / sql

Query your data using familiar SQL or intuitive Piped Processing Language (PPL)
https://opensearch.org/docs/latest/search-plugins/sql/index/
Apache License 2.0
116 stars 134 forks source link

SQL ingestion and ops - standup #791

Open anirudha opened 2 years ago

anirudha commented 2 years ago

Is your feature request related to a problem? A clear and concise description of what the problem is, e.g. I'm always frustrated when [...]

What solution would you like? A clear and concise description of what you want to happen.

What alternatives have you considered? A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context? Add any other context or screenshots about the feature request here.

MaxKsyunz commented 2 years ago

August 30, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
convert_tz and datetime function @MitchellGale-BitQuill In Review None Refactor tests and support null for invalid input Address review feedback None
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review None Waiting for opensearch-project/sql#746 to be merged --- Code dependency on opensearch-project/sql#746
Add get UTC date/time functions @MitchellGale-BitQuill Paused paused paused Depends on PR opensearch-project/sql#754
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand Paused Yury is working on .NET release and addressing PR feedback ---- --- --
Support highlight function in PPL query engine opensearch-project/sql#636 @forestmvey In Progress - Finalize design. Make adjustments based on finalized design. -
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Progress - Investigation and test implementation for using DOT . notation for accessing nested fields. Fixing quotation behavior
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress - - Address PR feedback --
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progess Addressing PR feedback Implementation rework Keep reworking + update tests None Aug 31
SQL Ingest -- JDBC support opensearch-project/sql-jdbc#25 @MaxKsyunz In Progress Meeting to align ingest priorities Finalize ingest priorities
MaxKsyunz commented 2 years ago

August 31, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
convert_tz and datetime function @MitchellGale-BitQuill In Review None Increased test coverage and manual testing Finish addressing review feedback None
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand Paused None --- --- Waiting for opensearch-project/sql#746 to be merged
Add get UTC date/time functions @MitchellGale-BitQuill Paused paused paused Depends on PR opensearch-project/sql#754
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand Paused Yury is working on .NET release and addressing PR feedback ---- --- --
Support highlight function in PPL query engine opensearch-project/sql#636 @forestmvey In Review - - - -
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Progress - Fixed quotation behavior support multiple highlight requests in a single statement
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress - Investigated how to address PR feedback Discuss design proposal with reviewer --
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progess .NET release is higher priority Created POC for new design and discussed with reviewer implement new design None Aug 31
SQL Ingest -- JDBC support opensearch-project/sql-jdbc#25 @MaxKsyunz Paused Paused due a change in priorities - -
MaxKsyunz commented 2 years ago

September 01, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Generated integration test report with v1 engine disabled. Create list features missing from v2 SQL engine based on test report.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand Paused Yury is working on .NET release and addressing PR feedback        
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand Paused None     Waiting for opensearch-project/sql#746 to be merged  
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None code cleanup, PR cleanup, pass all GH actions Address any new PR comments.    
Wildcard demo @forestmvey In Progress None   Do.    
Wildcard add optional arguments @forestmvey In Progress None   Implement.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progress .NET release is higher priority Implement new design Code clean up and test rework None  
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress     Discuss design proposal with reviewer --  
convert_tz and datetime function @MitchellGale-BitQuill In Review None Added support for local time zone to be default in tests. Added null return for invalid datetime entry. Address any new PR comments. None  
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   N/A Merge and relevant PRs and start adding new functions. Depends on PR opensearch-project/sql#754  
MaxKsyunz commented 2 years ago

September 02, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Began analyzing test results of running integration tests on v2 engine only Create GitHub issue to capture discussion.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand Paused Yury is working on .NET release and addressing PR feedback Nothing.      
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand Paused None Nothing.   Waiting for opensearch-project/sql#746 to be merged  
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None Nothing. Address any new PR comments.    
Wildcard demo @forestmvey In Progress None   Do if time.    
Wildcard add optional arguments @forestmvey In Progress None Implement Code cleanup, add tests, create PR.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progress   Update tests Fix test coverage validation None  
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress     Discuss design proposal with reviewer --  
convert_tz and datetime function @MitchellGale-BitQuill In Review None Got PR to stage with new PRs requested, addressed many of the concerns around naming, missing javadocs and more Fix datetime invalid date being treated as last day of the month. (Feb 30 is a valid date, but when converted it treats it like it was Feb 28th). None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   Merged relevant PRs Will be working on different topic today Depends on PR opensearch-project/sql#754  
MaxKsyunz commented 2 years ago

September 06, 2022 Status Updates

Note: Monday, September 5th, was a stat holiday.

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Nothing. Create GitHub issue to capture discussion.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Progress Implementation done, need to update tests Nothing. Working on tests    
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand Paused None Nothing.   Waiting for opensearch-project/sql#746 to be merged  
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None Address PR feedback. Address any new PR comments.    
Wildcard demo @forestmvey In Progress None Nothing Do if time.    
Wildcard add optional arguments @forestmvey In Review None Nothing. Address PR feedback.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progress On review opensearch-project/sql#113 Finished Address PR review None  
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress   Responded to PR comment with results of addressing suggestions. Follow-up with Peng re: moving relevance functions from :core to :opensearch as a separate ticket. --  
convert_tz and datetime function @MitchellGale-BitQuill In Review None Addressed PR comments. Working on fixing date validation for invalid dates. Breaking up more tests and adding more doc test, tests. None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   Paused Paused Depends on PR opensearch-project/sql#754  
acarbonetto commented 2 years ago

September 07, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Nothing. Create GitHub issue to capture discussion.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Progress Implementation done, need to update tests Nothing. Working on tests    
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand Paused None Nothing.   Waiting for opensearch-project/sql#746 to be merged  
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None Address PR feedback. Address any new PR comments.    
Wildcard demo @forestmvey In Progress None Nothing Do if time.    
Wildcard add optional arguments @forestmvey In Review None Nothing. Address PR feedback.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progress On review #113 Finished Address PR review None  
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz In Progress   Responded to PR comment with results of addressing suggestions. Follow-up with Peng re: moving relevance functions from :core to :opensearch as a separate ticket. --  
convert_tz and datetime function @MitchellGale-BitQuill In Review None Added invalid date test cases. Working on date validation. Working on fixing date validation for invalid dates. None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   Paused Paused Depends on PR opensearch-project/sql#754  
acarbonetto commented 2 years ago

Updates for September 8th

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Updated GitHub issue with test results. Rework error handling for easier review.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Progress Implementation done, need to update tests Working on tests Doctest, code style fixes, publish PR    
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In queue None Waiting for opensearch-project/sql#746 to be merged      
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None complete PR. Nothing.    
Wildcard demo @forestmvey In Progress None Nothing Finish.    
Wildcard add optional arguments @forestmvey In Review None Address PR feedback. Cleanup PR, request review.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Progress On review #113 Address PR review Merge PR if no changes requested None  
Refactor implementation of relevance queries opensearch-project/sql#746 @MaxKsyunz Done       --  
convert_tz and datetime function @MitchellGale-BitQuill In Review None Finished date validation. Tidied up test cases. Adding more doc tests and adding descriptions None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   Paused Paused Depends on PR opensearch-project/sql#754  
acarbonetto commented 2 years ago

Updates for September 9th

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Reworked error handling for easier review. Review new and publish results.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review Ready for review #114 Finished development Address PR feedback    
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review Ready for review #108 Finished development      
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None Nothing. Nothing.    
Wildcard demo @forestmvey In Progress None Nothing Finish.    
Highlight add optional arguments @forestmvey In Review None Address PR feedback. Minor revisions.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Review On review #113 Address PR review Merge PR if no changes requested None  
convert_tz and datetime function @MitchellGale-BitQuill In Review None finalized addressing PR reviews. Tidy up code then submit for new PR. Address new feedback. None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   Paused Paused Depends on PR opensearch-project/sql#754  
Prototype using aliases in SELECT clauses @Andrew Carbonetto In Progress Just starting investigation N/A Test/Code Investigation    
MitchellGale commented 2 years ago
Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Reviewed join, tophit and histogram related failures. Continue reviewing results    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review Ready for review #114        
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review Ready for review #108        
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review None Nothing. Nothing.    
Wildcard demo @forestmvey In Progress None Began. Finish.    
Highlight add optional arguments @forestmvey In Review None Address PR feedback. Cleanup IT tests, Submit PR when feedback for PPL syntax.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Review On review #113   Address PR review None  
convert_tz and datetime function @MitchellGale-BitQuill In Review None finalized addressing PR reviews. Get new PR comments and address. None  
Add get UTC date/time functions @MitchellGale-BitQuill Paused   N/A rebase off upstream main and build test cases. Depends on PR opensearch-project/sql#754  
Prototype using aliases in SELECT clauses @Yury Fridlyand In Progress Just starting investigation N/A Test/Code Investigation    
acarbonetto commented 2 years ago
Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Reviewed v2 parser integration test failures Compile and share results. Review uncategoried failures (20 tests)    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review On team review #114 Address PR feedback.      
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review On team review #108 Address PR feedback.      
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review On team review #112 Address PR feedback. Merge PR and raise upstream PR.    
Wildcard demo @forestmvey In Progress   Created draft demo Edit and post.    
Highlight add optional arguments @forestmvey Internal review approved, Cleanup and upstream PR needed   Cleanup IT tests, rework syntax for PPL to conform with other commands. Finish cleanup, make upstream PR.    
Update Jetty dependency to latest version @forestmvey in-progress   Nothing. Begin task.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Review On team review #113 Address PR review      
convert_tz and datetime function @MitchellGale-BitQuill In Review On team review #109 Addressed PR comments. Move strict date validation to exprdatetimevalue.    
Add get UTC date/time functions @MitchellGale-BitQuill Paused - waiting on convert_tz and datetime function     rebase off upstream main and build test cases. Depends on PR opensearch-project/sql#754  
Prototype using aliases in SELECT clauses @Yury-Fridlyand Done   Investigation report      
MitchellGale commented 2 years ago
Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress   Finished review and published results. Cross reference with existing issues.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review On team review #114 Address PR feedback.      
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review On team review #108 Address PR feedback.      
Support for wildcard in highlight opensearch-project/sql#636 @forestmvey In Review On upstream review opensearch-project/sql#827 Nothing. Nothing.    
Wildcard demo @forestmvey Complete.   Post demo: Support highlighting function in SQL/PPL query engine · Issue opensearch-project/sql#636 · opensearch-project/sql Nothing    
Highlight add optional arguments @forestmvey In Review On upstream review opensearch-project/sql#827 Finished cleanup, make PR to upstream. Nothing.    
Update Jetty dependency to latest version @forestmvey in-progress   Nothing. Begin task.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Review On team review #113 Address PR review      
convert_tz and datetime function @MitchellGale-BitQuill In Review On team review #109 Move strict date validation to exprdatetimevalue. Tidy up final concerns.    
Add get UTC date/time functions @MitchellGale-BitQuill Paused - waiting on convert_tz and datetime function     Start on implementation Depends on PR opensearch-project/sql#754  
Reorganize github workflows @Yury-Fridlyand In Progress   Workflow cleanup, reactivate forgotten ones Update the rest workflows    
Yury-Fridlyand commented 2 years ago

Topic | Dev | Status | Comments | What have you done yesterday? | What are you going to do today? | Any impediments? | ETA -- | -- | -- | -- | -- | -- | -- | -- V1/V2 Engine Gap Analysis / Integration Tests | @MaxKsyunz | In Progress |   | Finished review and published results. | Cross reference with existing issues. |   |   Add FROM_UNIXTIME, UNIX_TIMESTAMP functions | @Yury-Fridlyand | In Review | On team review [#114](https://github.com/Bit-Quill/opensearch-project-sql/pull/114) | Address PR feedback. |   |   |   Revise Error Reporting Improvements PR to Utilize QueryContext | @Yury-Fridlyand | In Review | On team review [#108](https://github.com/Bit-Quill/opensearch-project-sql/pull/108)/[#116](https://github.com/Bit-Quill/opensearch-project-sql/pull/116) | Address PR feedback. |   |   |   Support for wildcard in highlight opensearch-project/sql#636 | @forestmvey | In Review | On upstream review opensearch-project/sql#827 | Nothing. | Nothing. |   |   Wildcard demo | @forestmvey | Complete. |   | Post demo: [link](https://github.com/opensearch-project/sql/issues/636#issuecomment-1245958074) | Nothing |   |   Highlight add optional arguments | @forestmvey | In Review | On upstream review opensearch-project/sql#827 | Finished cleanup, make PR to upstream. | Nothing. |   |   Update Jetty dependency to latest version | @forestmvey | In Progress |   | Update Jetty and fix some errors raised | Fix remaining compilation errors and issues due to deprecated API usage |   |   Datetime NOW function opensearch-project/sql#754 | @Yury-Fridlyand | In Review | On upstream review opensearch-project/sql#754 | Merged requested changes |   |   |   convert_tz and datetime function | @MitchellGale-BitQuill | In Review | On team review [#109](https://github.com/Bit-Quill/opensearch-project-sql/pull/109) | finished ticket | Rebase and merge |   |   Add get UTC date/time functions | @MitchellGale-BitQuill | Paused - waiting on convert_tz and datetime function |   | Start on implementation | Add new functions into plugin | Depends on PR opensearch-project/sql#754 |   Reorganize github workflows | @Yury-Fridlyand | In Review | On team review [#118](https://github.com/Bit-Quill/opensearch-project-sql/pull/118) | Done |   |   |   Add/update date/time calculation functions | @Yury-Fridlyand | In Progress |   |   | Just started |   |  
acarbonetto commented 2 years ago

September 16, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress Updated github ticket with results opensearch-project/sql#787   Cross reference with existing issues. See project    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835 Publish PR to upstream      
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review On team review #108/#116 Review done Publish PR to upstream    
Update Jetty dependency to latest version @forestmvey In Progress   Out of office Fix remaining compilation errors and issues due to deprecated API usage. Determine if further work is needed if tests not passing.    
Datetime NOW function opensearch-project/sql#754 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#754        
convert_tz and datetime function @MitchellGale-BitQuill In Review On team review #109 finished review Publish PR to upstream    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress     continue to add new functions into plugin Depends on PR opensearch-project/sql#754  
Reorganize github workflows @Yury-Fridlyand In Review On team review #118 Review done Publish PR to upstream    
Add/update date/time calculation functions @Yury-Fridlyand In Progress     Update DATE_ADD function    
Relevance search and highlight demo @MaxKsyunz In Progress   Found sample data and demo outline Finalize demo script, record video    
Highlight, refactor PPL operator to function @Forest Vey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready.        
Yury-Fridlyand commented 1 year ago

September 19, 2022 Status Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz In Progress Updated github ticket with results opensearch-project/sql#787 Cross reference with existing issues. See project Plan next step.    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835 Publish PR to upstream Address PR feedback    
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Publish PR to upstream      
Update Jetty dependency to latest version @forestmvey Complete.   Assessed test failures and groomed follow up ticket. Nothing.    
Resolve EqualsAndHash warnings with new team member. @forestmvey In Progress     Implement fix and test.    
Datetime NOW function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#754        
convert_tz and datetime function @MitchellGale-BitQuill In Review On team review #109 finished review Publish PR to upstream    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress     continue to add new functions into plugin Depends on PR opensearch-project/sql#754  
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837 Publish PR to upstream      
Add/update date/time calculation functions @Yury-Fridlyand In Progress   Update DATE_ADD function Update DATE_SUB function    
Relevance search and highlight demo @MaxKsyunz In Progress   Finalize demo script, record video Edit and publish the demo    
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready.   Begin if time is available.    
acarbonetto commented 1 year ago

September 20, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz Done Updated github ticket with results opensearch-project/sql#787 Cross reference with existing issues. See project      
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835 Apply fix requested on review      
Revise Error Reporting Improvements PR to Utilize QueryContext @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update Jetty dependency to latest version @forestmvey Complete   Assessed test failures and groomed follow up ticket. Nothing.    
Resolve EqualsAndHash warnings with new team member. @forestmvey Complete   Resolve. Nothing.    
Update MacOS version for GHA @forestmvey In Progress   Begin      
Datetime NOW function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#754 Apply fix requested on review      
convert_tz and datetime function @MitchellGale-BitQuill In Review On team review #109 Pushed PR to upstream opensearch-project/sql#848 Nothing    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Added utc_date, utc_time, utc_timestamp Add implementation for utc_date, utc_time, utc_timestamp.    
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837 Add new test job      
Add/update date/time calculation functions @Yury-Fridlyand In Progress   Update DATE_SUB function Add DATEDIFF function    
Relevance search and highlight demo @MaxKsyunz Done   Edited the demo      
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready.   Begin if time is available.    
acarbonetto commented 1 year ago

September 22, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
V1/V2 Engine Gap Analysis / Integration Tests @MaxKsyunz Done Updated github ticket with results opensearch-project/sql#787 Cross reference with existing issues. See project At OpenSearch Conference    
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions (#722) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835        
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update Jetty dependency to latest version @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
Resolve EqualsAndHash warnings with new team member. @forestmvey Complete Upstream PR merged opensearch-project/sql#847        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
Datetime NOW function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#754        
convert_tz and datetime function @MitchellGale-BitQuill In Review On upstream review opensearch-project/sql#848   Address PR comments    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Added functionality for utc functions Adding tests for UTC_Date, UTC_time    
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837        
Add/update date/time calculation functions @Yury-Fridlyand In Progress Functions in scope of this task:TIMEDIFF, DATE_ADD, ADDDATE, DATE_SUB, SUBDATE, DATEDIFF, PERIOD_ADD, PERIOD_DIFF, ADDTIME, SUBTIME Finished implementation for all functions, added unit tests for some of them Continue with tests    
Relevance search and highlight demo @MaxKsyunz Done   Edited the demo      
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Continued implementation. Working on PPL portion with testing.    
Yury-Fridlyand commented 1 year ago

September 23, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions (#722) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835        
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update Jetty dependency to latest version @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
Resolve EqualsAndHash warnings with new team member. @forestmvey Complete Upstream PR merged opensearch-project/sql#847        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
Datetime NOW function @Yury-Fridlyand Completed opensearch-project/sql#754 merged        
convert_tz and datetime function @MitchellGale-BitQuill In Review On upstream review opensearch-project/sql#848   Address PR comments    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Added functionality for utc functions Adding tests for UTC_Date, UTC_time    
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837        
Add/update date/time calculation functions @Yury-Fridlyand In Progress Functions in scope of this task: TIMEDIFF, DATE_ADD, ADDDATE, DATE_SUB, SUBDATE, DATEDIFF, PERIOD_ADD, PERIOD_DIFF, ADDTIME, SUBTIME Working on unit tests Remove ambiguous signatures, continue with tests    
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Working on PPL portion with testing. PPL revisions and testing.    
MaxKsyunz commented 1 year ago

September 26, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions (#722) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835 rebased      
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update Jetty dependency to latest version @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
Resolve EqualsAndHash warnings with new team member. @forestmvey Complete Upstream PR merged opensearch-project/sql#847        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
convert_tz and datetime function @MitchellGale-BitQuill In Review On upstream review opensearch-project/sql#848   Address PR comments Blocked by opensearch-project/sql#722  
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Added functionality for utc functions Adding tests for UTC_Date, UTC_time    
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837        
Add/update date/time calculation functions @Yury-Fridlyand In Progress Functions in scope of this task:TIMEDIFF, DATE_ADD, ADDDATE, DATE_SUB, SUBDATE, DATEDIFF, PERIOD_ADD, PERIOD_DIFF, ADDTIME, SUBTIME Finished unit tests Documentation, integration tests Blocked by opensearch-project/sql#848 and opensearch-project/sql#722  
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. PPL revisions and testing. Updating test suite.    
Update Jetty in JDBC driver to address CVE @MaxKsyunz In Progress     Find non-vulenrable Jetty and Wiremock versions, create a branch.    
acarbonetto commented 1 year ago

September 27, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions (#722) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#835 rebased      
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update Jetty dependency to latest version @forestmvey@MaxKsyunz In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Find non-vulnerable Jetty and Wiremock versions, create a branch. Validate the branch with security scanner.    
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR        
convert_tz and datetime function @MitchellGale-BitQuill In Review On upstream review opensearch-project/sql#848 Addressed PR comments Rebase and make last PR change that was blocked by 835 (now merged) Blocked by opensearch-project/sql#722  
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Adding tests for UTC_Date, UTC_time Add variable tests to cover that testing current time might change during test, adds some margin of variation in result.    
Reorganize github workflows @Yury-Fridlyand In Review On upstream review opensearch-project/sql#837        
Add/update date/time calculation functions @Yury-Fridlyand In Progress Blocked by opensearch-project/sql#722 - Implementation done, but require minor fix after rebaseIT blocked by opensearch-project/sql#856/#853 or opensearch-project/sql#848 - all done, but few tests with DATETIME don’t work without it Finished doctest and integration tests Code clean-up & self-review and publish PR    
Highlight, refactor PPL operator to function @forestmvey In Progress Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Updating test suite. Cleanup code and make PR.    
acarbonetto commented 1 year ago

September 27, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Add FROM_UNIXTIME, UNIX_TIMESTAMP functions (#722) @Yury-Fridlyand Completed opensearch-project/sql#835          
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839   Address PR feedback      
Update Jetty dependency to latest version @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Validate the branch with security scanner. Debug failing test after dependency upgrade.      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR          
convert_tz and datetime function @MitchellGale-BitQuill In Review On upstream review opensearch-project/sql#848 Rebased and make last PR change that was blocked by opensearch-project/sql#835 (now merged) push and wait for new comments. Move formatter to separate file for date time Blocked by opensearch-project/sql#722    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Adding tests for UTC_Date, UTC_time Replace Date and Time UTC with call and parse of UTC_TIMESTAMP      
Reorganize github workflows @Yury-Fridlyand Completed opensearch-project/sql#837 Address PR feedback        
Add/update date/time calculation functions @Yury-Fridlyand In Review Blocked by opensearch-project/sql#722 - Implementation done, but require minor fix after rebaseIT blocked by opensearch-project/sql#856/#853 or opensearch-project/sql#848 - all done, but few tests with DATETIME don’t work without it On team review #122 Code clean-up & self-review and publish PR   Blocked by opensearch-project/sql#722, opensearch-project/sql#856/#853 or opensearch-project/sql#848    
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Cleanup code and make PR. Revise if necessary for PR comments.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress Just started   Analyze and investigate the scope      
v2 and v1 engine gap analysis @MaxKsyunz In Progress     Prepare for triage session      
acarbonetto commented 1 year ago

September 29, 2022

note: 2 Developers are off today; the team is off on the 30th;

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839          
Update Jetty dependency to latest version @forestmvey In Review initial investigation is complete; next steps is to implement the solution and raise a PR Debug failing test after dependency upgrade. Fixed issue with failing tests and create internal PR.      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR          
convert_tz and datetime function @MitchellGale-BitQuill Completed opensearch-project/sql#848 Move formatter to separate file for date time, rebase        
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Replace Date and Time UTC with call and parse of UTC_TIMESTAMP 29.09 - vacation, 30.09 - holiday      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 Rebase and update, since blockers resolved        
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Minor PR revisions. Finish off minor revisions based on PR comments, and merge internal PR. Should complete by end of day.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress Just started          
v2 and v1 engine gap analysis @MaxKsyunz In Progress     Prepare for triage session      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address PR feedback, rebase, publish        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress   Studied antlr4 and finished writing tests Change Lexer and Parser to implement fix    
acarbonetto commented 1 year ago

Status Report - October 3rd, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR          
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Vacation Push PR and add doctests      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 Rebase and update, since blockers resolved        
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Minor PR revisions. Cleanup PR, address any comments.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress Just started          
v2 and v1 engine gap analysis @MaxKsyunz In Progress     Prepare for triage session      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address PR feedback, rebase, publish        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress   Studied antlr4 and finished writing tests Change Lexer and Parser to implement fix      
Yury-Fridlyand commented 1 year ago

October 4th, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Address PR feedback        
Update Jetty dependency to latest version @forestmvey Complete initial investigation is complete; next steps is to implement the solution and raise a PR PR merged.        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR          
Add get UTC date/time functions @MitchellGale-BitQuill In Progress   Vacation Push PR and add doctests      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 Update docs        
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Cleanup PR, address any comments. Cleanup PR, address any comments.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress Just started          
v2 and v1 engine gap analysis @MaxKsyunz Done     Prepare for triage session      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address PR feedback        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress   Implemented Fix (Fix failed some checks) Fix failing checks      
Yury-Fridlyand commented 1 year ago

October 5rd, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839          
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR          
Add get UTC date/time functions @MitchellGale-BitQuill In Progress On team review #127 Push PR and add doctests Move over tests to use nowLikeFunctions test suite      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122          
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Cleanup PR, address any comments. Remove PPL portion, plan way forward with PPL, update PR.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress            
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address PR feedback        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress   Investigate failing tests Continued investigation      
Fix CI workflows in 1.x @Yury-Fridlyand In Review On upstream review opensearch-project/sql#888 Finished this fix        
Update docs for datetime functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#884          
Bump OpenSearch version to 3.x @MaxKsyunz In Progress Bringing this PR to completion.   Review results, flag next steps.      
acarbonetto commented 1 year ago

October 6th, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA  
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839          
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR   Begin implementing fix.      
Add get UTC date/time functions @MitchellGale-BitQuill In Progress On team review #127 Moved over tests to use nowLikeFunctions test suite Fix IT tests for PPL tests to use UTC time. Ask for PR review again.      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122   Address PR review      
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Remove PPL portion, plan way forward with PPL, update PR. Discuss PPL portion.      
Fix/add date/time comparison functions @Yury-Fridlyand In Progress   Implementation, started tests Fix test coverage, integration test      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address PR feedback        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress   Investigated possible solutions Bring up blocking issues at talk about next steps      
Fix CI workflows in 1.x @Yury-Fridlyand Done opensearch-project/sql#888          
Update docs for datetime functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#884          
Bump OpenSearch version to 3.x @MaxKsyunz In Progress Bringing this PR to completion.   Review results, flag next steps.      
acarbonetto commented 1 year ago

October 7, 2022

note: Monday is a holiday

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR      
Add get UTC date/time functions @MitchellGale-BitQuill In Progress On team review #127 Moved over tests to use nowLikeFunctions test suite Fix IT tests for PPL tests to use UTC time. Ask for PR review again.  
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #130 #131 #132 Split PR into 4 smaller ones    
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Remove PPL portion, plan way forward with PPL, update PR. Discuss PPL portion.  
Fix/add date/time comparison functions @Yury-Fridlyand In Progress On team review #129 Finished unit tests and coverage fixes Integration test  
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867    
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion opensearch-project/sql#128 Started implementing potential fix (with potentially breaking changes that need to be reviewed) Finish implementation and discuss next steps  
Update docs for datetime functions @Yury-Fridlyand In Review On upstream review opensearch-project/sql#884    
Bump OpenSearch version to 3.x @MaxKsyunz In Progress Fix IT tests for this PR Review results, investigate failures:BWC – ml-commons 2.4.0 plugin is not available.Gradle fails to build on macOS intel because it can’t find opensearch.pluginzip gradle plugin Find out what’s the status of 2.4 SNAPSHOTS is. note: resolving to use 2.3 as a fallback.    
acarbonetto commented 1 year ago

October 11, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839    
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR   resolve dependency errors to fix compilation errors.    
Add get UTC date/time functions @MitchellGale-BitQuill In Progress On team review #127 Moved over tests to use nowLikeFunctions test suite Fix IT tests for PPL tests to use UTC time. Ask for PR review again.    
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #130 #131 #132   Address review feedback    
Highlight, refactor PPL operator to function @forestmvey In Review Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Remove PPL portion, plan way forward with PPL, update PR. Discuss PPL portion.    
Fix/add date/time comparison functions @Yury-Fridlyand In Progress On team review #129 Integration tests for PPL ~1/2 scope Continue with integration tests    
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867    
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Started implementing potential fix (with potentially breaking changes that need to be reviewed) Finish implementation and discuss next steps    
Update docs for datetime functions @Yury-Fridlyand Done opensearch-project/sql#884 merged    
Bump OpenSearch version to 3.x @MaxKsyunz In Progress Fix IT tests for this PR Build OpenSearch locally to get opensearch.pluginzip Investigate if opensearch-sql 2.4 plugin is built already by CI – need to look into BWC test failures.  
Yury-Fridlyand commented 1 year ago

October 12, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Debugging compilation errors. Find aws-sdk-cpp version that doesn’t have broken googletest dependency.    
Add get UTC date/time functions @MitchellGale-BitQuill In Review On team review #127 Cleaned up code based on PR comments      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #130 #131 #132 Address review feedback      
Highlight, refactor PPL operator to function @forestmvey Complete Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. PR Merged. Create follow up issue for PPL portion. Update original highlight issue.    
Fix/add date/time comparison functions @Yury-Fridlyand In Progress On team review #129 Continue with integration testsReview other PRs Continue with integration tests    
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Implemented potential fix (with potentially breaking changes that need to be reviewed) Clean up and discuss implementation    
Bump OpenSearch version to 3.x @MaxKsyunz In Progress   Updated mapping json files to resolve unit test failures Run integration tests and review test results    
Update Date function @MitchellGale-BitQuill In Progress #126 opensearch-project/sql#709 opensearch-project/sql#580 Also makes Date eval as strict. Started ticket work Added scope to fix out of range date issue by using strict. Adding fix.    
Yury-Fridlyand commented 1 year ago

October 13, 2022

October 14: department meeting all day

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Find aws-sdk-cpp version that doesn’t have broken googletest dependency. Planning way forward to allow aws-c-common dependency with MacOS version 11 or 12.    
Add get UTC date/time functions @MitchellGale-BitQuill In Review On team review #127 Cleaned up code based on PR comments      
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #130 #131 #132 Address review feedback      
Highlight, refactor PPL operator to function @forestmvey Complete Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Nothing. Create follow up issue for PPL portion. Update original highlight issue.    
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review #129 Complete integration test, code clean up      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Implemented potential fix (with breaking changes that need to be reviewed) Discuss implementation    
Bump OpenSearch version to 3.x @MaxKsyunz In Progress   Updated mapping json files to resolve unit test failures Run integration tests and review test results    
Update Date function @MitchellGale-BitQuill In Progress #134 opensearch-project/sql#709 opensearch-project/sql#580 Also makes Date eval as strict. Started ticket work Added scope to fix out of range date issue by using strict. Adding fix.    
Rework timestamp function @Yury-Fridlyand In Progress just started   update TIMESTAMP, add tests    
acarbonetto commented 1 year ago

Oct 17, 2022 Updates

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Resolve dependencies using vcpkg for odbc driver as fix. Fixing compilation errors after updating aws-sdk-cpp dependency through vcpkg.    
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #130 #131 #132 Address review feedback      
Highlight, refactor PPL operator to function @forestmvey Complete Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Create follow up issue for PPL portion. Update original highlight issue.      
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review #129 Complete integration test, code clean up      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Investigated root cause of failing tests Continued investigation    
Bump OpenSearch version to 3.x @MaxKsyunz In Progress   Updated mapping json files to resolve unit test failures Run integration tests and review test results    
Update Date function @MitchellGale-BitQuill In Progress #134 opensearch-project/sql#709 opensearch-project/sql#580 Also makes Date eval as strict. Added null case. cherry pick PRs and cont work on finishing fix.      
Rework timestamp function @Yury-Fridlyand In Progress   update TIMESTAMP add tests    
Enable passing SQL plugin tests @GumpacG In progress Almost ready for a PR to upstream. Just need to investigate ML plugin test. Ran legacy integration tests and added issues regarding failing tests. Create a PR upstream once the ML plugin test has been resolved.    
SQL CLI fix for session token @Yury-Fridlyand In Review PR opensearch-project/sql#918 fixes opensearch-project/sql#854        
Yury-Fridlyand commented 1 year ago

October 19 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839   Revert combining error messages from V1 and V2    
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Fixing compilation errors after updating aws-sdk-cpp dependency through vcpkg. Fix failing odbc windows build.    
Add/update date/time calculation functions @Yury-Fridlyand In Review On team review #122 #131 #132 Address review feedback      
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 #130        
Highlight, refactor PPL operator to function @forestmvey Complete Upstream review changed scope to make PPL highlight to a fetch function. Refactoring and will update opensearch-project/sql#827 when ready. Create follow up issue for PPL portion. Update original highlight issue.      
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review #129 Address review feedback      
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867   Address review feedback    
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Completed Draft PR with implemented fix and updated description.      
Bump OpenSearch version to 3.x @MaxKsyunz In Progress   Updated mapping json files to resolve unit test failures Run integration tests and review test results    
Update Date function @MitchellGale-BitQuill In Progress #134 opensearch-project/sql#709 opensearch-project/sql#580 Also makes Date eval as strict. Added null case. cherry pick PRs and cont work on finishing fix.      
Rework timestamp function @Yury-Fridlyand In Progress     add more tests, next - IT    
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
SQL CLI fix for session token @Yury-Fridlyand In Review PR opensearch-project/sql#918 fixes opensearch-project/sql#854        
Update OpenSearch JDBC Driver version @GumpacG In Progress   Updated the hardcoded versions in different locations Create a property to sync the versions in different locations in order to avoid inconsistent versions    
Yury-Fridlyand commented 1 year ago

October 20, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Revert combining error messages from V1 and V2      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Fixing compilation errors after updating aws-sdk-cpp dependency through vcpkg. Fix failing odbc windows build.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review #122 #131 #132 Blocked, require YetAnotherQueryContext feature to operate properly (#937)        
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 #130        
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review #129 Blocked, require YetAnotherQueryContext feature to operate properly (#937)        
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address review feedback      
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Completed Draft PR with implemented fix and updated description.      
Bump OpenSearch version to 3.x @MaxKsyunz Blocked integration and unit tests passdoctest needs ml-commons 3.0.0        
Update Date function @MitchellGale-BitQuill In Progress #134 opensearch-project/sql#709 opensearch-project/sql#580 Also makes Date eval as strict .Blocked, require YetAnotherQueryContext feature to operate properly (#937) worked on addressing PR comments Finished update - waiting on PR comments    
Rework timestamp function @Yury-Fridlyand In Progress Depends on #132, which is blocked   add more tests, next - IT    
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
Update OpenSearch JDBC Driver version @GumpacG In Progress On upstream PR opensearch-project/sql-jdbc#34        
Fix jackson-databind CVE @GumpacG In progress     Update version to 2.13.4.2 or 2.14.0.rc2    
Add QUERY function @GabeFernandez310 In Progress QUERY seems to already belong to the function repository. Need to clarify if it’s ok to change as it may break things. Investigated code and relevant PRs Write tests    
Query context for built-in functions @MaxKsyunz In Progress   Implemented QueryContext and submitted a draft PR opensearch-project/sql#937 Address PR feed backInvestigate FunctionDSL unit test coverage    
acarbonetto commented 1 year ago

Status Report: October 24, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Revert combining error messages from V1 and V2      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Fix failing odbc windows build. Continuing fix for windows build, fixing CI build outputs, fixing vcpkg environment variables being deprecated in GH runner.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122 opensearch-project/sql#131 opensearch-project/sql#132; Blocked, require YetAnotherQueryContext feature to operate properly (#937)        
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 opensearch-project/notifications#211        
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review opensearch-project/sql#129; Blocked, require YetAnotherQueryContext feature to operate properly (#937)        
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address review feedback      
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion opensearch-project/sql#128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134 opensearch-project/sql#709 opensearch-project/sql#580; Also makes Date eval as strict. Blocked, require YetAnotherQueryContext feature to operate properly (#937) worked on addressing PR comments Finished update - waiting on PR comments    
Rework timestamp function @Yury-Fridlyand In Review On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked Finish tests and docs      
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
Add QUERY function @GabeFernandez310 @GumpacG In Progress   Wrote tests Implement changes to lexer and parser    
Query context for built-in functions @MaxKsyunz In Progress   Created POC for providing query context during execution of query physical plan. Refactor to POC to improve design and publish draft PR.    
JDBC release workflow @Yury-Fridlyand In Review opensearch-project/sql#140 #957Can’t test it - need git hooks and access to Jenkins First PoC done      
acarbonetto commented 1 year ago

Status Update: Oct 25, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Revert combining error messages from V1 and V2      
Update MacOS version for GHA @forestmvey In Progress initial investigation is complete; next steps is to implement the solution and raise a PR Fix failing odbc windows build. Continuing fix for windows build, fixing CI build outputs, fixing vcpkg environment variables being deprecated in GH runner.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937)        
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 opensearch-project/notifications#211        
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937)        
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address review feedback      
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion opensearch-project/sql#128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; Blocked, requires YetAnotherQueryContext feature to operate properly (#937) worked on addressing PR comments Finished update - waiting on PR comments    
Rework timestamp function @Yury-Fridlyand In Review On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked Finish tests and docs      
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress   Fixed implementation fro Parser and Lexer for Query function. Implemented Query function for Query builder. Add Integration Tests    
Query context for built-in functions @MaxKsyunz In Progress Two draft PRs up for review: opensearch-project/sql#960 & opensearch-project/sql#937 Created POC for providing query context during execution of query physical plan. Refactor to POC to improve design and publish draft PR.    
JDBC release workflow @Yury-Fridlyand In Review opensearch-project/sql#140 opensearch-project/sql#957; Can’t test it - need git hooks and access to Jenkins   Address PR feedback    
Fix aggregation for datetime types opensearch-project/sql#645 opensearch-project/sql#817 @Yury-Fridlyand In Progress   Create PoC Add UT, IT    
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432   In Progress Draft PR up for internal discussion Create PoC Discuss strategy with team    
acarbonetto commented 1 year ago

Status Update: Oct 26th, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839 Revert combining error messages from V1 and V2      
Update MacOS version for GHA @forestmvey In Review initial investigation is complete; next steps is to implement the solution and raise a PR Continuing fix for windows build, fixing CI build outputs, fixing vcpkg environment variables being deprecated in GH runner.      
Single-field relevance queries accept any qualified name as field parameter. @forestmvey In Progress     Ramp up on beginning task.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937)        
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 opensearch-project/notifications#211        
Fix/add date/time comparison functions @Yury-Fridlyand In Review On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937)        
Add TYPEOF function @Yury-Fridlyand In Review On upstream review opensearch-project/sql#867 Address review feedback      
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion opensearch-project/sql#128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; Blocked, requires YetAnotherQueryContext feature to operate properly (#937) worked on addressing PR comments Finished update - waiting on PR comments    
Rework timestamp function @Yury-Fridlyand In Review On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked Finish tests and docs      
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress   Fixed implementation fro Parser and Lexer for Query function. Implemented Query function for Query builder. Add Integration Tests    
Query context for built-in functions @MaxKsyunz In Progress Away yesterday.   Address PR feedback and prepare PR for upstream    
JDBC release workflow @Yury-Fridlyand In Review opensearch-project/sql#140 opensearch-project/sql#957; Can’t test it - need git hooks and access to Jenkins   Address PR feedback    
Fix aggregation for datetime types opensearch-project/sql#645 opensearch-project/sql#817 @Yury-Fridlyand In Progress   Create PoC Add UT, IT    
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR up for internal discussion Update PR with Tests    
Yury-Fridlyand commented 1 year ago

October 27th 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update MacOS version for GHA @forestmvey In Review On team review #136 Continuing fix for windows build, fixing CI build outputs, fixing vcpkg environment variables being deprecated in GH runner.      
Single-field relevance queries accept any qualified name as field parameter. @forestmvey In Progress     Ramp up on beginning task.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review #122; #131; #132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Add functions PERIOD_ADD and PERIOD_DIFF @Yury-Fridlyand In Review On upstream review opensearch-project/sql#933 Update function signature      
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review #129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress #134; #709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality   Push PR to upstream    
Rework timestamp function @Yury-Fridlyand Blocked On team review #139; Depends on #132, which is blocked Finish tests and docs      
Enable passing SQL plugin tests @GumpacG In Review On upstream PR opensearch-project/sql#926        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review #143 Fixed implementation for Parser and Lexer for Query function. Implemented Query function for Query builder. Add Integration Tests    
Query context for built-in functions @MaxKsyunz In Progress 2 PoC on upstream review opensearch-project/sql#937 and opensearch-project/sql#960   Address PR feedback    
JDBC release workflow @Yury-Fridlyand In Review On upstream review opensearch-project/sql#957; Can’t test it - need git hooks and access to Jenkins Address PR feedback      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review #144 Add IT Fix failing tests    
Fix for opensearch-project/sql#817 @Yury-Fridlyand In Review On team review #145 Finished Add more tests    
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR #142 up for internal discussion Create PoC Discuss strategy with team    
Yury-Fridlyand commented 1 year ago

October 31, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand In Review On upstream review opensearch-project/sql#839        
Update MacOS version for GHA @forestmvey In Review On upstream review opensearch-project/sql#987 Revise base on PR feedback. Address any new comments.    
Single-field relevance queries accept any qualified name as field parameter. @forestmvey In Progress   Ramp up on beginning task. Begin Implementation.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review #122; #131; #132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review #129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress #134; #709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality https://github.com/opensearch-project/sql/pull/991      
Rework timestamp function @Yury-Fridlyand Blocked On team review #139; Depends on #132, which is blocked Finish tests and docs      
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review #143 Finalized addressing PR comments. Fix failing integration tests    
Query context for built-in functions @MaxKsyunz In Progress 2 PoC on upstream review opensearch-project/sql#937 and opensearch-project/sql#960. opensearch-project/sql#937 was chosen as the way forward. Addressing feedback for opensearch-project/sql#937 Adding unit tests to FunctionDSL. They are required to refactor it as per feedback.    
JDBC release workflow @Yury-Fridlyand In Review On upstream review opensearch-project/sql#957; Can’t test it - need git hooks and access to Jenkins Address PR feedback      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review #144 Fix failing tests Fix failing tests    
Fix for opensearch-project/sql#817 @Yury-Fridlyand In Review On team review #145 Finish implementation Update PR and publish    
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR #142 up for internal discussion Create PoC Discuss strategy with team    
Support timestamp format {ts 'YYYY-MM-DD hh:mm:ss'} opensearch-project/sql#364 @GumpacG In Progress   Investigated the issue Implement fix    
Update Jackson Databind to 2.13.4.2 opensearch-project/sql#992 @margarit-h Done Internal PR #148 Implemented the changes and created a PR for internal review. Create a PR to upstream.    
Fix for opensearch-project/sql#613 @forestmvey In Progress   Investigation started      
Yury-Fridlyand commented 1 year ago

October 32, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Revise Error Reporting Improvements PR to Utilize QueryContext (#838) @Yury-Fridlyand Completed opensearch-project/sql#839 Backport is pending in opensearch-project/sql#1008        
Update MacOS version for GHA @forestmvey Complete. Merged upstream opensearch-project/sql#987        
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Progress   Out of office. Implement tests/make PR.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review #122; #131; #132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review #129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 In Progress PoC on team review for discussion #128 Completed Draft PR with implemented fix and updated description.      
Update Date function @MitchellGale-BitQuill In Progress #134; #709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality opensearch-project/sql#991      
Rework timestamp function @Yury-Fridlyand Blocked On team review #139; Depends on #132, which is blocked Fix function to throw an error on incorrect format instead of returning NULL      
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review #143 Fixed failing tests and checkstyle errors Internal PR up for review    
Query context for built-in functions @MaxKsyunz In Progress     Addressing feedback from internal review.Publish upstream PR.    
JDBC release workflow @Yury-Fridlyand In Review On upstream review opensearch-project/sql#957; Can’t test it - need git hooks and access to Jenkins   Add uploading to S3    
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review #144 Fix failing tests      
Fix for opensearch-project/sql#817 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1000 Publish to upstream      
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR #142 up for internal discussion   Update internal PR with tests    
Update Jackson Databind to 2.13.4.2 opensearch-project/sql#992 @margarit-h Done Internal PR #148 Implemented the changes and created a PR for internal review. Create a PR to upstream.    
Investigate if SPACE and SUBSTRING_INDEX are supported in the legacy engine opensearch-project/sql#570 and opensearch-project/sql#574 @GumpacG In Progress   Investigate if supported in legacy engine      
Fix opensearch-project/sql#998 @Yury-Fridlyand In Review On team review #151        
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review #152        
acarbonetto commented 1 year ago

Status Update: November 2, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Progress   Implement tests/make PR. Internal discussion on implementation and continue implementing tests.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality   Rebase off 2.4 and implement the changes Peng and Chen requested.    
Rework timestamp function @Yury-Fridlyand Blocked On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review opensearch-project/sql#143 Finished addressing issues with internal PR, Rebasing off of 2.x, squashed commits. Raise PR upstream    
Query context for built-in functions @MaxKsyunz In Progress     Addressing feedback from internal review.Publish upstream PR.    
JDBC release workflow @Yury-Fridlyand Paused Draft PR opensearch-project/sql#957        
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review opensearch-project/sql#144        
Aggregation functions return invalid result when processing NULLs opensearch-project/sql#817 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1000 Publish to upstream      
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR opensearch-project/sql#142 up for internal discussion   Update internal PR with tests    
Add WILDCARDQUERY function @GumpacG In Progress   Investigated Make a plan for implementation and start execution    
Casting a DOUBLE as a FLOAT truncates all decimal digits opensearch-project/sql#998 @Yury-Fridlyand In Review On team review opensearch-project/sql#151        
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review opensearch-project/sql#152        
Disable loading ML-commons doctests @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1011        
Yury-Fridlyand commented 1 year ago

September 67 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review.   Internal discussion on implementation and continue implementing tests. Address PR.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review #122; #131; #132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review #129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion #128       Unknown - need to determine scope
Update Date function @MitchellGale-BitQuill In Progress #134; #709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality Rebase off 2.4 and implement the changes Peng and Chen requested.      
Rework timestamp function @Yury-Fridlyand Blocked On team review #139; Depends on #132, which is blocked        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review #143 Finished addressing issues with internal PR, Rebasing off of 2.x, squashed commits. Raise PR upstream    
Query context for built-in functions @MaxKsyunz In Progress   Merge with upstream – there were a lot of conflicts. Publish upstream PR.    
JDBC release workflow @Yury-Fridlyand Paused Draft PR opensearch-project/sql#957        
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review #144        
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR #142 up for internal discussion   Update internal PR with tests    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review #156 Created integration tests and doctests. Add support for parameters.      
Casting a DOUBLE as a FLOAT truncates all decimal digits opensearch-project/sql#998 @Yury-Fridlyand Completed opensearch-project/sql#1025        
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review #152        
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Progress PoC available at #154 Fix done Make tests    
Add Support For Legacy Query Functions @GabeFernandez310 In Progress Draft PR for alternate syntax for match function #157 Began writing unit tests and implementation for 'matchphrasequery' Continue working on ‘matchphrasequery’ and address comments on draft PR    
MitchellGale commented 1 year ago

November 8th, 2022


Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review.   Address PR. Discuss on sync-up for how to handle Qualified Names in relevance functions.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality   Addressed Chen’s comments    
Rework timestamp function @Yury-Fridlyand Blocked On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review opensearch-project/sql#143 Finished addressing issues with internal PR, Rebasing off of 2.x, squashed commits. Raise PR upstream    
Query context for built-in functions @MaxKsyunz In Progress   Merge with upstream – there were a lot of conflicts. Publish upstream PR.    
JDBC release workflow @Yury-Fridlyand Paused Draft PR opensearch-project/sql#957        
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Progress On team review opensearch-project/sql#144        
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR opensearch-project/sql#142 up for internal discussion   Update internal PR with tests    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Addressing internal PR comments      
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review opensearch-project/sql#152        
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Progress PoC available at opensearch-project/sql#154 IT done Add UT    
Add Support For Legacy Query Functions @GabeFernandez310 In Progress Draft PR for alternate syntax for match function opensearch-project/sql#157 Addressed comments on Draft PR. Refactored code to remove duplicate code and fix checkstyle errors. Continue working on ‘matchphrasequery’ unit tests and implementation    
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking. Added some support for escapes. Finishing fix and starting on test writing.    
MitchellGale commented 1 year ago

November 9th, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review.   Discuss on sync-up for how to handle Qualified Names in relevance functions. Pushed revisions. Address any new PR feedback and sprint support work.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality Merged to upstream!      
Rework timestamp function @Yury-Fridlyand Blocked On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review opensearch-project/sql#143 Finished addressing issues with internal PR, Rebasing off of 2.x, squashed commits. Raise PR upstream    
Query context for built-in functions @MaxKsyunz In Progress   Merge with upstream – there were a lot of conflicts. Publish upstream PR.    
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061        
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto In Progress Draft PR opensearch-project/sql#142 up for internal discussion   Update internal PR with tests    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Addressing internal PR comments Handle cases for escaping wildcard characters    
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review opensearch-project/sql#152        
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Review On team review opensearch-project/sql#154 Rework, add tests      
Add Support For Legacy Query Functions @GabeFernandez310 In Progress Draft PR for alternate syntax for match function opensearch-project/sql#157 Implemented matchphrasequery Work on integration tests for matchphrasequery and move on to next function    
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking. Finishing fix and starting on test writing. Finish writing tests    
acarbonetto commented 1 year ago

November 16th, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067        
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Progress   finding what is supported by legacy parser and updating with any new features. Adding optional parameter support for quoted keys.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked On team review opensearch-project/sql#129; Blocked, requires YetAnotherQueryContext feature to operate properly (#937/#960)        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Update Date function @MitchellGale-BitQuill In Progress opensearch-project/sql#134; opensearch-project/sql#709; opensearch-project/sql#580; Also makes Date eval as strict; unblocked by removing partial functionality Merged to upstream!      
Rework timestamp function @Yury-Fridlyand Blocked On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Add QUERY function @GabeFernandez310 & @GumpacG In Progress On team review opensearch-project/sql#143 Finished addressing issues with internal PR, Rebasing off of 2.x, squashed commits. Raise PR upstream    
Query context for built-in functions @MaxKsyunz In Progress   Merge with upstream – there were a lot of conflicts. Publish upstream PR.    
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Investigate metadata field usage in SELECT/WHERE clauses opensearch-project/sql#639 opensearch-project/sql#432 @acarbonetto Completed PoC PR opensearch-project/sql#142 up for comment and linked in opensearch-project/sql#639   Fixed PR and pushed comment on Issue    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Address final PR comments Fix broken LIKE tests    
Resolve Prometheus dependency for IT on non-Linux OS @Yury-Fridlyand In Review On team review opensearch-project/sql#152        
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Review On team review opensearch-project/sql#154   Investigate possibility and necessity to fix legacy and PPL parsers too    
Add Support For Legacy Query Functions @GabeFernandez310 In Progress 2 PRs are being reviewed internally Addressed comments on Internal PRs. Fixed and added Tests. Wait for review and address comments before making PR upstream.    
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking. Finishing fix and starting on test writing. Finish writing tests    
Add engine switch option @Yury-Fridlyand In Progress On team review opensearch-project/sql#160; does not include ODBC Finish fixes and tests      
Add CBRT to the V2 engine @Margarit Hakobyan In Progress On team review opensearch-project/sql#166 Implement the solution and tests Test/verify, address PR review comments    
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Progress   Working on PoC and tests for it Keep working    
acarbonetto commented 1 year ago

Status update - Friday November 18th

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067   Waiting for feedback    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Internal Review On internal review - opensearch-project/sql#168 Raised PR for internal review      
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Fixed broken LIKE tests Add more tests for LIKE    
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Review On team review opensearch-project/sql#154   Investigate possibility and necessity to fix legacy and PPL parsers too    
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090        
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC Finish fixes and tests     Stalled - this cannot go into public API
Add CBRT to the V2 engine @Margarit Hakobyan In Reivew On upstream review opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169 PoC finished TBD    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress   Continued investigation of tickets Continue investigation of tickets    
Add Support For *_of_year functions @GabeFernandez310 In ProgressIn review - opensearch-project/sql#1089   Investigated and began implementation for day_of_year function Make internal PR and begin writing tests and implementation for the next function    
MaxKsyunz commented 1 year ago

Status Update -- Tuesday November 22, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. @forestmvey In Progress     Ramp up on task.    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067 Attempt moving fields analysis execution to more appropriate classes. Waiting on feedback.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Internal Review On internal review - opensearch-project/sql#168 Addressing PR comments. Waiting on feedback.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Implemented conversion of text field to keyword when using the LIKE function Add tests for both LIKE and wildcard_query for text and keyword    
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Review On team review opensearch-project/sql#154   Investigate possibility and necessity to fix legacy and PPL parsers too    
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090        
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC Finish fixes and tests     Stalled - this cannot go into public API
Add CBRT to the V2 engine @Margarit Hakobyan Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169 PoC finished TBD    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Look into [FEATURE] Add new data type for textENHANCEMENT    
Add Support For *_of_year functions @GabeFernandez310 In Progress   Made internal draft PR for month_of_year function Continue testing and implementation for month_of_year and week_of_year function    
Make now function constant within a DQL statement @MaxKsyunz In Review   Merge with upstream refactoring of DSL and BuiltInFunctionRepository classes. Propose several ways to integrate FunctionProperties with refactored DSL and BuiltinFunctionRepository classes    
Pagination in v2 SQL engine @MaxKsyunz In Progress   Prepare specification document Discuss specification and implementation consideration with AWS team    
Add CBRT to the PPL @Margarit Hakobyan Internal review opensearch-project/sql#171   Update, rebase, test internal PR Get Internal PR reviewed and merged, create an upstream PR    
acarbonetto commented 1 year ago

Status update - Nov 23, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. @forestmvey In Progress   Ramp up on task. Continue scoping out changes needed to support in V2 engine.    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067 Waiting on feedback. Discuss way forward with team to allow migration to opensearch module.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Internal Review On internal review - opensearch-project/sql#168 Minor PR revisions. Address PR comments and continue review.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156 Add tests for both LIKE and wildcard_query for text and keyword Create a PR to upstream if there are no more comments    
Fix for opensearch-project/sql#650 @Yury-Fridlyand In Review On team review opensearch-project/sql#154   Investigate possibility and necessity to fix legacy and PPL parsers too    
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090        
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC Finish fixes and tests     Stalled - this cannot go into public API
Add CBRT to the V2 engine @margarit-h Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169 PoC finished TBD    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Added new mapping for type for integration tests Finish adding integration tests for Issue opensearch-project/sql#1038. Put PR up for comment.  
Add Support For *_of_year functions @GabeFernandez310 In Progress All PRs are up internally Made PRs for all functions. Address PR comments before making PR upstream.    
Make now function constant within a DQL statement @MaxKsyunz In Review   Proposed several ways to integrate FunctionProperties with refactored DSL and BuiltinFunctionRepository classes create a revised PR    
Pagination in v2 SQL engine @MaxKsyunz In Progress   Discussed specification and implementation consideration with AWS team Revise proposal based on feedback now function takes higher priority.  
Add CBRT to the PPL @margarit-h Upstream review opensearch-project/sql#1097   Update, rebase, test internal PR Create an upstream PR    
Yury-Fridlyand commented 1 year ago

Movember 24

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. @forestmvey In Progress   Continue scoping out changes needed to support in V2 engine. Testing support with legacy engine. Listing existing IT tests to be ported to V2.    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067 Discuss way forward with team to allow migration to opensearch module. Rework implementation to use new visitor to aid going forward with moving relevance functions to opensearch module.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Review On internal review - opensearch-project/sql#168 Minor PR revisions. Address PR comments make upstream PR.    
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156   Create a PR to upstream if there are no more comments    
Fix for opensearch-project/sql#650 @Yury-Fridlyand Paused On team review #154Waiting for 3.x branch        
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090, opensearch-project/sql#1103 Made PR upstream     Done - Waiting for reviews
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC       Stalled - this cannot go into public API
Add CBRT to the V2 engine @margarit-h Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169        
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Finished adding integration tests for Issue opensearch-project/sql#1038. Put PR up for comment. Work on JDBC data correctness  
Add Support For *_of_year functions @GabeFernandez310 In Progress All PRs are up internally Addressed some comments internally Continue addressing feedback on internal PR   Before end of sprint
Make now function constant within a DQL statement @MaxKsyunz In Review   Proposed several ways to integrate FunctionProperties with refactored DSL and BuiltinFunctionRepository classes create a revised PR    
Pagination in v2 SQL engine @MaxKsyunz In Progress   Discussed specification and implementation consideration with AWS team Revise proposal based on feedback now function takes higher priority.  
Add CBRT to the PPL @margarit-h Upstream review opensearch-project/sql#1097   Address PR review feedback Address more feedback if available or get it merged.    
Run v2 vs legacy correctness tests for opensearch-project/sql#787 @GumpacG Internal review   Ran comparison tests for sanity_integration_tests.txt and tableau_integration_tests.txt      
Add position() function to the V2 engine. @margarit-h In Progress   Start implementing SQL portion Continue implementing SQL part and start implementing PPL support.    
acarbonetto commented 1 year ago

Status Update - Nov 25, 2022

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. @forestmvey In Progress   Testing support with legacy engine. Listing existing IT tests to be ported to V2. Groom follow up tickets, make branch with IT/Parser tests, Plan implementation.    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067   Rework implementation to aid going forward with moving relevance functions to opensearch module.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Review On upstream review - opensearch-project/sql#168 Address PR comments make upstream PR.      
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061   Address PR feedback    
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In Review Internal review opensearch-project/sql#156   Create a PR to upstream if there are no more comments    
Fix for opensearch-project/sql#650 @Yury-Fridlyand Paused On team review #154Waiting for 3.x branch        
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090, opensearch-project/sql#1103 Made PR upstream     Done - Waiting for reviews
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC       Stalled - this cannot go into public API
Add CBRT to the V2 engine @margarit-h Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169   Address PR feedback    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Finished adding integration tests for Issue opensearch-project/sql#1038. Put PR up for comment. Work on JDBC data correctness  
Add Support For *_of_year functions @GabeFernandez310 In Progress All PRs are up internally Addressed some comments internally Continue addressing feedback on internal PR   Before end of sprint
Make now function constant within a DQL statement @MaxKsyunz Blocked Waiting from comments from other maintainers create a revised PR      
Pagination in v2 SQL engine @MaxKsyunz In Progress   Discussed specification and implementation consideration with AWS team Revise proposal based on feedback now function takes higher priority.  
Add CBRT to the PPL @margarit-h Upstream review opensearch-project/sql#1097   Address PR review feedback Address more feedback if available or get it merged.    
Run v2 vs legacy correctness tests for opensearch-project/sql#787 @GumpacG Internal review   Ran comparison tests for sanity_integration_tests.txt and tableau_integration_tests.txt      
Add position() function to the V2 engine. @margarit-h In ProgressWIP internal PR: opensearch-project/sql#177   Continue implementation, add integration tests. Continue implementation, add unit tests and docs.    
Yury-Fridlyand commented 1 year ago

Nov 28

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. @forestmvey@GumpacG In Progress   Testing support with legacy engine. Listing existing IT tests to be ported to V2. Groom follow up tickets, make branch with IT/Parser tests, Plan implementation.    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067   Rework implementation to aid going forward with moving relevance functions to opensearch module.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Review On upstream review - opensearch-project/sql#168 Address PR comments make upstream PR.      
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061        
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In review Upstream review opensearch-project/sql#1108   Create a PR to upstream if there are no more comments    
Fix for opensearch-project/sql#650 @Yury-Fridlyand Paused On team review #154Waiting for 3.x branch        
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090, opensearch-project/sql#1103 Made PR upstream     Done - Waiting for reviews
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC       Stalled - this cannot go into public API
Add CBRT to the V2 engine @margarit-h Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169 Related/dependent on opensearch-project/sql#1038   Rework according to PR comments    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Finished adding integration tests for Issue opensearch-project/sql#1038. Put PR up for comment. Work on JDBC data correctness  
Add Support For *_of_year functions @GabeFernandez310 In Progress All PRs are up internally Added Tests Continue addressing feedback on internal PR and make PR upstream   Before end of sprint
Make now function constant within a DQL statement @MaxKsyunz Blocked Waiting from comments from other maintainers create a revised PR      
Pagination in v2 SQL engine @MaxKsyunz In Progress   Discussed specification and implementation consideration with AWS team Revise proposal based on feedback now function takes higher priority.  
Add CBRT to the PPL @margarit-h Upstream review opensearch-project/sql#1097   Address PR review feedback Address more feedback if available or get it merged.    
Add position() function to the V2 engine. @margarit-h In ProgressInternal PR: opensearch-project/sql#177   Continue adding tests, address PR review comments Start working on adding PPL support    
Yury-Fridlyand commented 1 year ago

Nov 29

Topic Dev Status Comments What have you done yesterday? What are you going to do today? Any impediments? ETA
Spike on porting NESTED/REVERSE_NESTED queries from legacy engine. opensearch-project/sql#1111 @forestmvey@GumpacG In Progress     Implement parser changes for nested by using Highlight implementation as a reference    
relevance queries accept any qualified name as field parameter.#613 @forestmvey In Review On upstream review opensearch-project/sql#1067   Rework implementation to aid going forward with moving relevance functions to opensearch module.    
Update MULTI_MATCH to support legacy alternate syntax. @forestmvey In Review On upstream review - opensearch-project/sql#168 Address PR comments make upstream PR.      
Add/update date/time calculation functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#122; opensearch-project/sql#131; opensearch-project/sql#132; Blocked by opensearch-project/sql#1047        
Fix/add date/time comparison functions @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#129; Blocked by opensearch-project/sql#1047        
Fix bug where multiplication in queries is treated as alias @GabeFernandez310 Paused PoC on team review for discussion opensearch-project/sql#128       Unknown - need to determine scope
Rework timestamp function @Yury-Fridlyand Blocked by opensearch-project/sql#1047 On team review opensearch-project/sql#139; Depends on opensearch-project/sql#132, which is blocked        
Query context for built-in functions @MaxKsyunz In review opensearch-project/sql#1047   Merging conflicts      
Fix aggregation for datetime types opensearch-project/sql#645 @Yury-Fridlyand In Review On upstream review opensearch-project/sql#1061        
Add WILDCARDQUERY function opensearch-project/sql#1032 @GumpacG In review Upstream review opensearch-project/sql#1108   Address upstream PR comments    
Fix for opensearch-project/sql#650 @Yury-Fridlyand Paused On team review #154Waiting for 3.x branch        
Add Support For Legacy Query Functions @GabeFernandez310 In review opensearch-project/sql#1090 In review - opensearch-project/sql#1090, opensearch-project/sql#1103 Made PR upstream     Done - Waiting for reviews
Added escape character support in 3.x @MitchellGale-BitQuill In Progress Putting in 3.x branch because change is breaking.        
Add engine switch option @Yury-Fridlyand Paused On team review opensearch-project/sql#160; does not include ODBC       Stalled - this cannot go into public API
Add CBRT to the V2 engine @margarit-h Done Merged opensearch-project/sql#1081        
Fix for opensearch-project/sql#794 @Yury-Fridlyand In Review On team review opensearch-project/sql#169 Related/dependent on opensearch-project/sql#1038   Rework according to PR comments    
Investigation and scope of data correctness tickets @MitchellGale-BitQuill In Progress     Found bug with misreported success Finish fix on bug  
Add Support For *_of_year functions @GabeFernandez310 In Progress All PRs are up internally Refactored Test Code. Addressed PR Feedback Make PR upstream once re-reviewing is done internally   Before end of sprint
Make now function constant within a DQL statement @MaxKsyunz Blocked Waiting from comments from other maintainers create a revised PR      
Pagination in v2 SQL engine @MaxKsyunz In Progress   Discussed specification and implementation consideration with AWS team Revise proposal based on feedback now function takes higher priority.  
Add CBRT to the PPL @margarit-h Merged opensearch-project/sql#1097   Address PR review feedback Address more feedback if available or get it merged.    
Add position() function to the V2 engine. @margarit-h In ProgressInternal PR: opensearch-project/sql#177   Address PR review comments, work on PPL support Continue working on adding PPL support    
Fix for opensearch-project/sql-jdbc#19 @Yury-Fridlyand In Progress PoC available opensearch-project/sql#178 Make PoC Test it