amirmohammadkz / personality_detection

BB-SVM model for automatic personality detection of the essays dataset (Big-Five personality labeled traits)
https://sentic.net/personality-detection-using-bagged-svm-over-bert.pdf
MIT License
29 stars 8 forks source link

Completed all the steps and I don't know how to interpret the results #2

Open python-ai-13 opened 3 years ago

python-ai-13 commented 3 years ago

I can see a new file called "svm_0_layer_with_mairesserev200.csv" with a bunch of numbers in it... if that's the results how do I read it?
Sorry, I'm new with python.

Thanks

saminfatehi commented 3 years ago

Each row in this file is about a personality trait (EXT, NEU, AGR, CON, and OPN). The 10 first columns show the accuracy of 10 fold cross-validation and the last column shows the average accuracy in 10 folds.

If it is helpful for you, you can make us happy by giving our project a star. :D

di-press commented 3 years ago

Hi @python-ai-13! Please, have you ever had any issue when trying to run the code? I'm having an error with bert-as-service. I type in the terminal, as suggested in this README:

bert-serving-start -model_dir uncased_L-12_H-768_A-12/ -num_worker=4 -max_seq_len=NONE -show_tokens_to_client -pooling_layer -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1

and my computer crashes instantly, and I need to reboot it manually. It crashes when WORKER-1 tries to load a graph. Maybe I don't have enough memory or other requirement to run the code... Please, do you have any suggestions? Thanks!

saminfatehi commented 3 years ago

Hi @di-press; Maybe you don't have enough memory for 4 workers. you can set num_worker parameter less than 4. like this:

bert-serving-start -model_dir uncased_L-12_H-768_A-12/ -num_worker=1 -max_seq_len=NONE -show_tokens_to_client -pooling_layer -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1

di-press commented 3 years ago

Thanks, @saminfatehir ! I'll try :)

Pompiia commented 3 years ago

Hi,

I'm in the same boat as python-ai13. If I understood the whole process well, the personality traits extracted are from the essay_200_max_split.csv (that itself comes from essays_original_splitted.csv), with each row inside this file corresponding to a different essay, written by different users. However, the final output (that is "svm_0_layer_with_mairesserev200.csv") only yield the accuracy of predicting well the personality of the users of the original dataset, but do not tell us any information concerning their personality (is user X possess the Extraversion type or else, that sort of thing)? Is there a Python script that would do such a thing? Thank you.

saminfatehi commented 3 years ago

Hi,

Since this is a supervised learning model, we evaluate our model on test data and report the accuracy. If you want to know the personality of a new text (base on our model), you can use predictor.py (which was recently added). Hope this helps!