asreview / asreview-makita

Workflow generator for simulation studies using the command line interface of ASReview LAB
https://asreview.ai
MIT License
30 stars 17 forks source link

Create script to calculate and save different TD and ATD values (scri… #22

Closed FioByr closed 1 year ago

FioByr commented 1 year ago

…pt_ATD_calculations.py.template)

feat(average_record_TD function): Add function to calculate average-record-TD. Compute an average for each row (TD values for a record across different simulation runs) and store them as a new column (average_record_TD)

feat(average_simulation_TD function): Add function to calculate average-simulation-TD. Compute an average for each column (TD values for all records across one simulation run) and store them in a new row (average_simulation_TD; each average value underneath the corresponding simulation column).

feat(get_TD_values function): Add function to create json file of TD values (average record TD, record ATD, average simulation TD, and simulation ATD).

feat(saving json files): Save json files in each simulation runs metrics folder.

jteijema commented 1 year ago

Great work! Discussion point: should this file be merged with the td merge file? @FioByr @J535D165

I like this file separate, but I do see some overlap with that file.

FioByr commented 1 year ago

Great work! Discussion point: should this file be merged with the td merge file? @FioByr @J535D165

I like this file separate, but I do see some overlap with that file.

Yes I am fine with it being merged, but I also think it can be separate.

jteijema commented 1 year ago

I have merged the atd calc and the merge td script into 1 single script. The output of the atd calculation has been added to the td files. Note: I am unsure about the column/row wise average calculations. I lack the mathematical proof, but preliminary results show that there appears to be a difference between the average record atd and the average simulation atd (likely as a consequence of missing values?).

J535D165 commented 1 year ago

Great, thanks all!