search

dnaaun / openFraming

Tools for automatic frame discovery and labeling based on topic modeling and deep learning, made widely accessible to researchers from non computational backgrounds.
http://www.openframing.org
9 stars 7 forks source link

support for multiple languages for BERT and LDA #341

Open asmithh opened 3 years ago

asmithh commented 3 years ago

BERT classifier & LDA topic modeling should support multiple languages.

asmithh commented 3 years ago

https://github.com/davidatbu/openFraming/tree/modeling/multilingual attempts to resolve, but I need some help/support on testing and hooking up all the piping before I can really be sure it works.

dnaaun commented 3 years ago

Hi @lentil-soup , this looks like a great start. Do you feel like you just need some pointers before doing more work? Or should I take this over the finish line?

asmithh commented 3 years ago

I'd love some pointers on how to continue & test this, particularly if taking it over the finish line will be nontrivial for you :) thanks!

On Mon, Dec 28, 2020, 3:06 PM davidatbu notifications@github.com wrote:

Hi @lentil-soup https://github.com/lentil-soup , this looks like a great start. Do you feel like you just need some pointers before doing more work? Or should I take this over the finish line?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/davidatbu/openFraming/issues/341#issuecomment-751848097, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO2OVYBUEBK2FAQERLKJ5XDSXDQMFANCNFSM4VMJMYKQ .

dnaaun commented 3 years ago

Thanks for this Alyssa.

Regarding how to test: these two commands should let you run the tests locally (beware, they might modify your local copy of the database backend).

docker-compose -f docker-compose-testing.yml build

docker-compose -f docker-compose-testing.yml up --abort-on-container-exit

If you get that working fine on the master branch, could you look into making sense of and modifying test_classifier.py and test_topic_model.py ?

In essence, the test cases emulate requests and then assert that the caused behavior and/or returned value makes sense. For any new request parameters, response parameters, we'd need to modify the test files to incorporate them.

asmithh commented 3 years ago

Hey David - Haven't found the time to do this - would you be okay picking it up? Don't want to leave it hanging forever, so I figured it would be better to be honest about what I can/can't do right now. Thanks! Alyssa

On Mon, Dec 28, 2020 at 3:35 PM davidatbu notifications@github.com wrote:

Thanks for this Alyssa.

Regarding how to test: these two commands should let you run the tests locally (beware, they might modify your local copy of the database backend).

docker-compose -f docker-compose-testing.yml build

docker-compose -f docker-compose-testing.yml up --abort-on-container-exit

If you get that working fine on the master branch, could you look into making sense of and modifying test_classifier.py https://github.com/davidatbu/openFraming/blob/master/services/web/backend/tests/api/test_classifier.py and test_topic_model.py https://github.com/davidatbu/openFraming/blob/master/services/web/backend/tests/api/test_topic_model.py ?

In essence, the test cases emulate requests and then assert that the caused behavior and/or returned value makes sense. For any new request parameters, response parameters, we'd need to modify the test files to incorporate them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/davidatbu/openFraming/issues/341#issuecomment-751856072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO2OVYERLTLCYPCPKK2YAI3SXDTYFANCNFSM4VMJMYKQ .

dnaaun commented 3 years ago

Sounds good Alyssa. Thanks for this great start. I'll take it from here.

On Sat, Jan 2, 2021 at 12:43 PM alyssa notifications@github.com wrote:

Hey David - Haven't found the time to do this - would you be okay picking it up? Don't want to leave it hanging forever, so I figured it would be better to be honest about what I can/can't do right now. Thanks! Alyssa

On Mon, Dec 28, 2020 at 3:35 PM davidatbu notifications@github.com wrote:

Thanks for this Alyssa.

Regarding how to test: these two commands should let you run the tests locally (beware, they might modify your local copy of the database backend).

docker-compose -f docker-compose-testing.yml build

docker-compose -f docker-compose-testing.yml up --abort-on-container-exit

If you get that working fine on the master branch, could you look into making sense of and modifying test_classifier.py < https://github.com/davidatbu/openFraming/blob/master/services/web/backend/tests/api/test_classifier.py

and test_topic_model.py < https://github.com/davidatbu/openFraming/blob/master/services/web/backend/tests/api/test_topic_model.py

?

In essence, the test cases emulate requests and then assert that the caused behavior and/or returned value makes sense. For any new request parameters, response parameters, we'd need to modify the test files to incorporate them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/davidatbu/openFraming/issues/341#issuecomment-751856072 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AO2OVYERLTLCYPCPKK2YAI3SXDTYFANCNFSM4VMJMYKQ

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/davidatbu/openFraming/issues/341#issuecomment-753506784, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMQIHCYACVYEZJJL6HTIX53SX5LKPANCNFSM4VMJMYKQ .