Develop medical text classification operator

[Make0913]

• [ Step1--collect some medical data like medical QAs, then use the data to train a NLP model and save the trained model]

• [Step2--develop a new operator like the sentiment analysis operator in Texera, the new operator will load the trained model and provide an interface to test data ]

• [ Step3--After the operator receive the test data, the trained model will give corresponding predictions, finally the operator output these predictions to the " view results" icon to visualize it]

[Make0913]

• [ First Workflow] Using existing model
• URL: https://texera.ics.uci.edu/workflow/116
  1. read raw data( json file )into Texera by converting the data format
  2. develop an operator to integrate Bert model
  3. use the operator to classify, we input tuples to the operator, and the operator output corresponding prediction

Bert pretrained model: https://github.com/google-research/bert

[Make0913]

The first workflow has been built successfully in pytorch environment, with the predict speed a little slow

[Make0913]

• [Second Workflow ] Retrain the model for medical text classification URL: https://texera.ics.uci.edu/workflow/116
  1. read raw data into Texera
  2. develop a Python UDF to filter the data to get high-frequency labels and split these filtered data into train set, valid set and test set
  3. develop a Python UDF for supervised training, after training, the UDF output predictions for test set

[Make0913]

The second workflow has finished, with small amount of training data

[Make0913]

For now my task of medical text classification has entirely completed

[Yicong-Huang]

@Make0913 please document the workflows, you can include screenshots, gifs, and even texera links here.

[Make0913]

I have archived my workflow and related data files to project account on Texera URL: https://texera.ics.uci.edu/workflow/116

I will show the screenshot of my work below: For the workflow of training step:

• the workflow structure

  

• the result

  

For the workflow of predict step:

• the workflow structure

  

• the result

  

[Yicong-Huang]

@Make0913 please do the same steps for this use case, 1) share workflows and files to project account. 2) add a reference here for each workflow. Thanks!

[Make0913]

Done!

[Yicong-Huang]

Thanks! I will archive this issue now.