search

Cyberjusticelab / JusticeAI

JusticeAI (ProceZeus) is a web chat bot that aims to facilitate access to judicial proceedings involving Quebec tenant/landlord law
https://cyberjusticelab.github.io/JusticeAI/docs/rendered/
MIT License
21 stars 16 forks source link

[#425] regex fixes #430

Closed TaimoorRana closed 6 years ago

TaimoorRana commented 6 years ago
  1. Added pronom to DATE REGEX
  2. Using multiple_words function instead of (le|la|les|aux..) in order to allow all types of pronoms to be captured
codecov[bot] commented 6 years ago

Codecov Report

Merging #430 into master will not change coverage. The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #430   +/-   ##
=======================================
  Coverage   70.96%   70.96%           
=======================================
  Files          54       54           
  Lines        2273     2273           
  Branches       24       24           
=======================================
  Hits         1613     1613           
  Misses        659      659           
  Partials        1        1
Impacted Files Coverage Δ
...ture_extraction/post_processing/regex/regex_lib.py 100% <100%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 156bafa...8b44890. Read the comment docs.

TaimoorRana commented 6 years ago

LATEST (THIS BRANCH)

INFO: Total precedents parsed: 40003 INFO: Total precedents with apartment_dirty : 913 INFO: Total precedents with asker_is_landlord : 29696 INFO: Total precedents with asker_is_tenant : 2830 INFO: Total precedents with bothers_others : 40 INFO: Total precedents with disrespect_previous_judgement : 126 INFO: Total precedents with landlord_inspector_fees : 508 INFO: Total precedents with landlord_notifies_tenant_retake_apartment : 437 INFO: Total precedents with landlord_pays_indemnity : 47 INFO: Total precedents with landlord_relocation_indemnity_fees : 513 INFO: Total precedents with landlord_rent_change : 2420 INFO: Total precedents with landlord_rent_change_doc_renseignements : 310 INFO: Total precedents with landlord_retakes_apartment : 1721 INFO: Total precedents with landlord_sends_demand_regie_logement : 132 INFO: Total precedents with tenant_sends_demand_regie_logement : 1 INFO: Total precedents with signed_proof_of_rent_debt : 29 INFO: Total precedents with rent_increased : 1426 INFO: Total precedents with tenant_continuous_late_payment : 4063 INFO: Total precedents with tenant_damaged_rental : 694 INFO: Total precedents with tenant_dead : 31 INFO: Total precedents with tenant_financial_problem : 307 INFO: Total precedents with tenant_group_responsability : 1256 INFO: Total precedents with tenant_individual_responsability : 3780 INFO: Total precedents with tenant_is_bothered : 27 INFO: Total precedents with tenant_lease_indeterminate : 1405 INFO: Total precedents with tenant_left_without_paying : 7119 INFO: Total precedents with tenant_monthly_payment : 34191 INFO: Total precedents with tenant_owes_rent : 21902 INFO: Total precedents with tenant_refuses_retake_apartment : 409 INFO: Total precedents with tenant_rent_not_paid_more_3_weeks : 20734 INFO: Total precedents with tenant_withold_rent_without_permission : 90 INFO: Total precedents with violent : 418 INFO: Total precedents with tenant_not_paid_lease_timespan : 145 INFO: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO: Total precedents with additional_indemnity_money : 13968 INFO: Total precedents with declares_resiliation_is_correct : 5477 INFO: Total precedents with landlord_serious_prejudice : 2547 INFO: Total precedents with orders_expulsion : 19223 INFO: Total precedents with orders_immediate_execution : 13657 INFO: Total precedents with orders_resiliation : 19831 INFO: Total precedents with orders_tenant_pay_first_of_month : 742 INFO: Total precedents with tenant_ordered_to_pay_landlord : 27088 INFO: Total precedents with tenant_ordered_to_pay_landlord_legal_fees : 26926 INFO: Total precedents with landlord_prejudice_justified : 19249 INFO: Total precedents with landlord_retakes_apartment_indemnity : 100 INFO: Total precedents with authorize_landlord_retake_apartment : 282 INFO: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO: Loading classifier_labels.bin INFO: classifier_labels.bin is successfully loaded INFO: Loading precedent_vectors.bin INFO: precedent_vectors.bin is successfully loaded INFO: Formatting data INFO: Training SVM classifier INFO: Sample size: 40003 INFO: Train size: 32002 INFO: Test size: 8001 INFO: Training Classifier Using Multi Class SVM INFO: Loading model_metrics.bin INFO: model_metrics.bin is successfully loaded INFO: Testing Classifier INFO: Classifier results:

INFO: Column: additional_indemnity_money INFO: Test accuracy: 79.56505436820397% INFO: Precision: [ 0.87372014 0.6851796 ] INFO: Recall: [ 0.79704223 0.79315164] INFO: F1: [ 0.83362165 0.73522267]

INFO: Column: declares_resiliation_is_correct INFO: Test accuracy: 91.3885764279465% INFO: Precision: [ 0.96064746 0.66535122] INFO: Recall: [ 0.93848832 0.76083032] INFO: F1: [ 0.94943861 0.70989474]

INFO: Column: landlord_serious_prejudice INFO: Test accuracy: 96.4379452568429% INFO: Precision: [ 0.99036374 0.66192733] INFO: Recall: [ 0.97150845 0.85510204] INFO: F1: [ 0.98084549 0.74621549]

INFO: Column: orders_expulsion INFO: Test accuracy: 91.98850143732034% INFO: Precision: [ 0.95244249 0.88939981] INFO: Recall: [ 0.88966683 0.95231925] INFO: F1: [ 0.91998502 0.91978476]

INFO: Column: orders_immediate_execution INFO: Test accuracy: 84.06449193850769% INFO: Precision: [ 0.92366412 0.72146119] INFO: Recall: [ 0.82640865 0.86813187] INFO: F1: [ 0.87233403 0.78802993]

INFO: Column: orders_resiliation INFO: Test accuracy: 93.63829521309836% INFO: Precision: [ 0.95244249 0.9213456 ] INFO: Recall: [ 0.91895262 0.95389627] INFO: F1: [ 0.93539789 0.93733842]

'precision', 'predicted', average, warn_for) INFO: Column: orders_tenant_pay_first_of_month INFO: Test accuracy: 98.20022497187851% INFO: Precision: [ 0.98200225 0. ] INFO: Recall: [ 1. 0.] INFO: F1: [ 0.99091941 0. ]

INFO: Column: tenant_ordered_to_pay_landlord INFO: Test accuracy: 85.02687164104486% INFO: Precision: [ 0.79529617 0.87239264] INFO: Recall: [ 0.71495693 0.91371397] INFO: F1: [ 0.75298969 0.89257532]

INFO: Column: tenant_ordered_to_pay_landlord_legal_fees INFO: Test accuracy: 89.78877640294964% INFO: Precision: [ 0.87827557 0.90612245] INFO: Recall: [ 0.79708477 0.94660734] INFO: F1: [ 0.83571285 0.92592257]

INFO: Column: landlord_prejudice_justified INFO: Test accuracy: 82.42719660042495% INFO: Precision: [ 0.82282353 0.82591309] INFO: Recall: [ 0.8426506 0.80446637] INFO: F1: [ 0.83261905 0.81504867]

INFO: Column: landlord_retakes_apartment_indemnity INFO: Test accuracy: 99.76252968378952% INFO: Precision: [ 0.9976253 0. ] INFO: Recall: [ 1. 0.] INFO: F1: [ 0.99881124 0. ]

INFO: Column: authorize_landlord_retake_apartment INFO: Test accuracy: 99.60004999375079% INFO: Precision: [ 0.99786325 0.66666667] INFO: Recall: [ 0.99811416 0.63829787] INFO: F1: [ 0.99798869 0.65217391]

INFO: Training multi output regression INFO: Size of dataset: 27086 2018-04-02 10:13:47.291798: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA INFO: R2: 0.81 INFO: Explained Variance: 0.81 INFO: Mean Absolute Error: 337.85 INFO: Mean Squared Error: 868862.55

TaimoorRana commented 6 years ago

OLD (commit #a5c58700 from March 26th. Before any regex changes were made)

INFO: Total precedents parsed: 40003 INFO: Total precedents with apartment_dirty : 913 INFO: Total precedents with asker_is_landlord : 29606 INFO: Total precedents with asker_is_tenant : 2699 INFO: Total precedents with bothers_others : 40 INFO: Total precedents with disrespect_previous_judgement : 126 INFO: Total precedents with landlord_inspector_fees : 508 INFO: Total precedents with landlord_notifies_tenant_retake_apartment : 437 INFO: Total precedents with landlord_pays_indemnity : 47 INFO: Total precedents with landlord_relocation_indemnity_fees : 513 INFO: Total precedents with landlord_rent_change : 2420 INFO: Total precedents with landlord_rent_change_doc_renseignements : 310 INFO: Total precedents with landlord_retakes_apartment : 1721 INFO: Total precedents with landlord_sends_demand_regie_logement : 132 INFO: Total precedents with tenant_sends_demand_regie_logement : 1 INFO: Total precedents with signed_proof_of_rent_debt : 29 INFO: Total precedents with rent_increased : 1426 INFO: Total precedents with tenant_continuous_late_payment : 4063 INFO: Total precedents with tenant_damaged_rental : 694 INFO: Total precedents with tenant_dead : 31 INFO: Total precedents with tenant_financial_problem : 307 INFO: Total precedents with tenant_group_responsability : 1256 INFO: Total precedents with tenant_individual_responsability : 3780 INFO: Total precedents with tenant_is_bothered : 26 INFO: Total precedents with tenant_lease_indeterminate : 1405 INFO: Total precedents with tenant_left_without_paying : 7115 INFO: Total precedents with tenant_monthly_payment : 34191 INFO: Total precedents with tenant_owes_rent : 21902 INFO: Total precedents with tenant_refuses_retake_apartment : 409 INFO: Total precedents with tenant_rent_not_paid_more_3_weeks : 20734 INFO: Total precedents with tenant_withold_rent_without_permission : 90 INFO: Total precedents with violent : 418 INFO: Total precedents with tenant_not_paid_lease_timespan : 112 INFO: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ INFO: Total precedents with additional_indemnity_money : 13968 INFO: Total precedents with declares_resiliation_is_correct : 5477 INFO: Total precedents with landlord_serious_prejudice : 2508 INFO: Total precedents with orders_expulsion : 19212 INFO: Total precedents with orders_immediate_execution : 13657 INFO: Total precedents with orders_resiliation : 19831 INFO: Total precedents with orders_tenant_pay_first_of_month : 742 INFO: Total precedents with tenant_ordered_to_pay_landlord : 26596 INFO: Total precedents with tenant_ordered_to_pay_landlord_legal_fees : 26926 INFO: Total precedents with landlord_prejudice_justified : 18320 INFO: Total precedents with landlord_retakes_apartment_indemnity : 100 INFO: Total precedents with authorize_landlord_retake_apartment : 270 INFO: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO: Loading classifier_labels.bin INFO: classifier_labels.bin is successfully loaded INFO: Loading precedent_vectors.bin INFO: precedent_vectors.bin is successfully loaded INFO: Formatting data INFO: Training SVM classifier INFO: Sample size: 40003 INFO: Train size: 32002 INFO: Test size: 8001 INFO: Training Classifier Using Multi Class SVM INFO: Loading model_metrics.bin INFO: model_metrics.bin is successfully loaded INFO: Testing Classifier INFO: Classifier results:

INFO: Column: additional_indemnity_money INFO: Test accuracy: 79.55255593050869% INFO: Precision: [ 0.8735338 0.68508454] INFO: Recall: [ 0.79704223 0.79280224] INFO: F1: [ 0.83353683 0.73501782]

INFO: Column: declares_resiliation_is_correct INFO: Test accuracy: 91.37607799025122% INFO: Precision: [ 0.96064162 0.6648265 ] INFO: Recall: [ 0.93834325 0.76083032] INFO: F1: [ 0.94936151 0.70959596]

INFO: Column: landlord_serious_prejudice INFO: Test accuracy: 96.42544681914761% INFO: Precision: [ 0.99063645 0.65664557] INFO: Recall: [ 0.9711321 0.85743802] INFO: F1: [ 0.98078732 0.7437276 ]

INFO: Column: orders_expulsion INFO: Test accuracy: 91.98850143732034% INFO: Precision: [ 0.95244249 0.88939981] INFO: Recall: [ 0.88966683 0.95231925] INFO: F1: [ 0.91998502 0.91978476]

INFO: Column: orders_immediate_execution INFO: Test accuracy: 84.06449193850769% INFO: Precision: [ 0.92348453 0.7215961 ] INFO: Recall: [ 0.82659837 0.86776557] INFO: F1: [ 0.8723596 0.78795942]

INFO: Column: orders_resiliation INFO: Test accuracy: 93.63829521309836% INFO: Precision: [ 0.95244249 0.9213456 ] INFO: Recall: [ 0.91895262 0.95389627] INFO: F1: [ 0.93539789 0.93733842]

'precision', 'predicted', average, warn_for) INFO: Column: orders_tenant_pay_first_of_month INFO: Test accuracy: 98.20022497187851% INFO: Precision: [ 0.98200225 0. ] INFO: Recall: [ 1. 0.] INFO: F1: [ 0.99091941 0. ]

INFO: Column: tenant_ordered_to_pay_landlord INFO: Test accuracy: 84.53943257092864% INFO: Precision: [ 0.8058337 0.86132539] INFO: Recall: [ 0.7006056 0.91677552] INFO: F1: [ 0.74954444 0.88818584]

INFO: Column: tenant_ordered_to_pay_landlord_legal_fees INFO: Test accuracy: 89.98875140607424% INFO: Precision: [ 0.8790932 0.90870262] INFO: Recall: [ 0.80322209 0.94660734] INFO: F1: [ 0.83944678 0.92726777]

INFO: Column: landlord_prejudice_justified INFO: Test accuracy: 79.99000124984377% INFO: Precision: [ 0.82267168 0.77407309] INFO: Recall: [ 0.80506329 0.79376368] INFO: F1: [ 0.81377225 0.78379473]

INFO: Column: landlord_retakes_apartment_indemnity INFO: Test accuracy: 99.76252968378952% INFO: Precision: [ 0.9976253 0. ] INFO: Recall: [ 1. 0.] INFO: F1: [ 0.99881124 0. ]

INFO: Column: authorize_landlord_retake_apartment INFO: Test accuracy: 99.66254218222723% INFO: Precision: [ 0.99811581 0.7 ] INFO: Recall: [ 0.99849208 0.65116279] INFO: F1: [ 0.99830391 0.6746988 ]

INFO: Training multi output regression INFO: Size of dataset: 26594 2018-04-01 20:44:22.703471: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

INFO: R2: 0.81 INFO: Explained Variance: 0.81 INFO: Mean Absolute Error: 377.99 INFO: Mean Squared Error: 863114.49