This PR removes the concept Environment. The latest ER-diagram of our system should be:
Here are graphs to show the before-after, also the delta:
In order to incorporate this ER diagram, several designs are implemented.
0. DDL Changes
Tables and columns related to environment are now removed
use `texera_db`;
ALTER TABLE `environment_of_workflow`
DROP FOREIGN KEY `environment_of_workflow_ibfk_2`;
ALTER TABLE `dataset_of_environment`
DROP FOREIGN KEY `dataset_of_environment_ibfk_1`;
ALTER TABLE `dataset_of_environment`
DROP FOREIGN KEY `dataset_of_environment_ibfk_2`;
ALTER TABLE `workflow_executions`
DROP FOREIGN KEY `workflow_executions_ibfk_3`;
-- Dropping the dependent tables
DROP TABLE IF EXISTS `environment_of_workflow`;
DROP TABLE IF EXISTS `dataset_of_environment`;
-- Dropping the environment table
DROP TABLE IF EXISTS `environment`;
ALTER TABLE `workflow_executions`
DROP COLUMN `environment_eid`;
1. File URI
To uniquely identify a file globally, the URI is now in the format of:
For example, for user texera@uci.edu, a file tweets/california.csv in dataset Twitter, version v3. The URI is:
/texera@uci.edu/Twitter/v3/tweets/california.csv
2. Workflow Sharing
When sharing a workflow to another user, only the files scanned by this workflow are visible to the users. The dataset access will NOT be propagated through this sharing
Demo
First of all, the environment tab is removed:
To scan a source file, users no need to add datasets explicitly to somewhere, user can select files from all accessible datasets and versions.
Future Work
Currently, datasets are loaded fully and frequently. This can result in the GUI not responsive and fluent enough. An ongoing effort led by one of our team member is to improve the file selection interaction logic. This PR can make the loading lazily happen, and make the search of datasets & file much more easier
@bobbai00 : the diagrams are very nice. Per our discussion, please add comments about how to improve the interface to allow a user to select a file (as a future PR).
This PR removes the concept
Environment
. The latest ER-diagram of our system should be:Here are graphs to show the before-after, also the delta:
In order to incorporate this ER diagram, several designs are implemented.
0. DDL Changes
Tables and columns related to
environment
are now removed1. File URI
To uniquely identify a file globally, the URI is now in the format of:
For example, for user
texera@uci.edu
, a filetweets/california.csv
in datasetTwitter
, versionv3
. The URI is:2. Workflow Sharing
When sharing a workflow to another user, only the files scanned by this workflow are visible to the users. The dataset access will NOT be propagated through this sharing
Demo
First of all, the environment tab is removed:
To scan a source file, users no need to add datasets explicitly to somewhere, user can select files from all accessible datasets and versions.
Future Work
Currently, datasets are loaded fully and frequently. This can result in the GUI not responsive and fluent enough. An ongoing effort led by one of our team member is to improve the file selection interaction logic. This PR can make the loading lazily happen, and make the search of datasets & file much more easier