dnaaun / openFraming

Tools for automatic frame discovery and labeling based on topic modeling and deep learning, made widely accessible to researchers from non computational backgrounds.
http://www.openframing.org
9 stars 7 forks source link

Create additional pretrained classifiers #296

Open dnaaun opened 4 years ago

monajalal commented 4 years ago

We need them for Media Frame Corpus (MFC) policy issues and potential addition of climate change and covid19 would make it more exciting. https://github.com/dallascard/media_frames_corpus

dnaaun commented 4 years ago

The following classifiers are now available in the backend(the below is an excerpt of GET /classifiers/). @edward-edberg , how are we doing in terms of the new two-page design for the classifiers part? If we have that figured out, we can add the below to it.

[
  {
    "classifier_id": 20,
    "classifier_name": "Tobacco",
    "trained_by_openFraming": false,
    "category_names": [
      "Economic",
      "Health and Safety",
      "Legality_Constitutionality_Jurisdiction",
      "Other",
      "Policy Prescription and Evaluation"
    ],
    "notify_at_email": "davidat@bu.edu",
    "status": "completed",
    "metrics": {
      "accuracy": 0.8170289855072463,
      "macro_f1_score": 0.8123707030890623,
      "macro_precision": 0.814152437148811,
      "macro_recall": 0.8115565105166219
    }
  },
  {
    "classifier_id": 21,
    "classifier_name": "Same-sex marriage",
    "trained_by_openFraming": false,
    "category_names": [
      "Legality_Constitutionality_Jurisdiction",
      "Morality",
      "Other",
      "Political",
      "Public Sentiment"
    ],
    "notify_at_email": "davidat@bu.edu",
    "status": "completed",
    "metrics": {
      "accuracy": 0.8367346938775511,
      "macro_f1_score": 0.8205516865894265,
      "macro_precision": 0.8199471120907219,
      "macro_recall": 0.8226336522640233
    }
  },
  {
    "classifier_id": 23,
    "classifier_name": "Immigration",
    "trained_by_openFraming": false,
    "category_names": [
      "Crime and Punishment",
      "Legality_Constitutionality_Jurisdiction",
      "Other",
      "Policy Prescription and Evaluation",
      "Political"
    ],
    "notify_at_email": "davidat@bu.edu",
    "status": "completed",
    "metrics": {
      "accuracy": 0.629940119760479,
      "macro_f1_score": 0.6236960368228012,
      "macro_precision": 0.6270172272868804,
      "macro_recall": 0.6214781894428739
    }
  }
]

Small note: trained_by_openFraming should be true in all cases above, but we're not using that in the frontend right now.

edward-edberg commented 4 years ago

Okay.

await axios
    .post(endpointPostTestName, {
        **test_set_name: testName,
        notify_at_email: email**
    })

I'm building the last page test.html. I need to transfer those inputs (test_set_name, notify_at_email). How can I get those? (dummy test set name? another input email which is not efficient?)

edward-edberg commented 4 years ago

Nevermind I know how to do it but now I need a link that you'll generate. Is it available?

edward-edberg commented 4 years ago

Oh, it is just a basic url (?classifier_id=). I believe it will be working today

dnaaun commented 4 years ago

@edward-edberg , yes, it will be exaclty that. I believe the page you made is called test.html, so the email will contain the link www.openframing.org/test.html?classifier_id=<classifier_id>. I will test out your branch this afternoon and let you know if it works for me too. After that, we'll wait for Yimeng's input on the dropdown, and then we'll merge.

Great work!

edward-edberg commented 4 years ago

@davidatbu yeah, that is the link. I've tested on my local machine. It is up and running.

dnaaun commented 4 years ago

Hi @edward-edberg , sorry for the delay. I tested this on my machine and it works! Great work. I have a few comments:

  1. Right now, when someone clicks on "framing" on the home page, they are still taken to the old framing page. I think the old framing page should be modified so that people can either

    1. Select one of the "pretrained" classifiers, or
    2. They can click on a button that takes them to www.openframing/train.html

That's it, nothing else should be on that page, I think.

  1. When loading www.openframing.org/test.html?classifier_id=, the page should display the name of the classifier that the user chose.

@monajalal included, what do you think about the two points above?