Cyberjusticelab / JusticeAI

JusticeAI (ProceZeus) is a web chat bot that aims to facilitate access to judicial proceedings involving Quebec tenant/landlord law
https://cyberjusticelab.github.io/JusticeAI/docs/rendered/
MIT License
21 stars 16 forks source link

[#350] additional indemnity money #361

Closed Samuel-Campbell closed 6 years ago

Samuel-Campbell commented 6 years ago

1) removed anti-regex because they are not used properly 1.1) Example: not_violent is not set to 1 if the violent regex does not match. However the NLP does so. Therefore the results are bad. 2) improved tenant pays landlord by changing loss function from mean_absolute_percentage_error to mean_squared error 3) Finished additional indemnity money 4) tested with and without using average values in a vector --> both yield very similar result during testing. Therefore using average column values is better when missing data 5) Each regressor now gets tested after training 6) METRICS: Tenant Pays Landlord: INFO: Size of dataset: 26594 INFO: R2: 0.70 INFO: Explained Variance: 0.70 INFO: Mean Absolute Error: 492.03 INFO: Mean Squared Error: 1362642.05 Additional Indemnity Money: INFO: R2: 0.78 INFO: Explained Variance: 0.78 INFO: Mean Absolute Error: 471.99 INFO: Mean Squared Error: 831383.89

6) I think the error is caused by the date regex. It's probably not working well

codecov[bot] commented 6 years ago

Codecov Report

Merging #361 into master will increase coverage by 0.4%. The diff coverage is 36.66%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #361     +/-   ##
=========================================
+ Coverage   68.33%   68.73%   +0.4%     
=========================================
  Files          51       52      +1     
  Lines        2021     2060     +39     
  Branches       16       16             
=========================================
+ Hits         1381     1416     +35     
- Misses        638      642      +4     
  Partials        2        2
Impacted Files Coverage Δ
...ture_extraction/post_processing/regex/regex_lib.py 100% <ø> (ø) :arrow_up:
src/ml_service/init.py 51.16% <ø> (ø) :arrow_up:
...ion/single_output_regression/abstract_regressor.py 60.6% <28.57%> (+32.48%) :arrow_up:
...n/single_output_regression/tenant_pays_landlord.py 37.93% <33.33%> (ø)
...le_output_regression/additional_indemnity_money.py 37.5% <37.5%> (ø)
...regression/multi_output/multi_output_regression.py 53.12% <37.5%> (-2.05%) :arrow_down:
...raining/classifier/multi_output/multi_class_svm.py 87.8% <66.66%> (-0.72%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f8224a4...4d9e6a6. Read the comment docs.