lkorczowski commented 3 years ago

Already included in the #98 , the sleep labels allowed (if present) to only analysis data during sleep.

Few improvement are made to:

[x] (improvement) convert the many type of different datetime format using a single read_sleep_file function (pytested)
[x] (new) drop any incorrect sleep labels (e.g. the empty first line)

Before merging, I need to validate with at least one file containing one of the following format:

This example was from Maxime's labels.

Horodatage;NumŽro d'Žpoque;Sommeil
[];[];
23:55:00;343;
23:55:30;344;ƒveil
23:56:00;345;ƒveil
23:56:30;346;ƒveil
23:57:00;347;ƒveil
23:57:30;348;ƒveil

Any type is converted using read_sleep_file to

Start Time;Sleep
23:55:30;Wake
23:56:00;Wake
23:56:30;Wake
23:57:00;Wake
23:57:30;Wake

codecov[bot] commented 3 years ago

Codecov Report

Merging #103 (fddfbb0) into master (deb2d75) will increase coverage by 0.01%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #103      +/-   ##
==========================================
+ Coverage   99.45%   99.46%   +0.01%     
==========================================
  Files          11       11              
  Lines         732      754      +22     
==========================================
+ Hits          728      750      +22     
  Misses          4        4

Impacted Files	Coverage Δ
tinnsleep/data.py	`100.00% <ø> (ø)`
tinnsleep/reports.py	`100.00% <ø> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update deb2d75...fddfbb0. Read the comment docs.

lkorczowski commented 3 years ago

Note : I'm adding manually the line "2TL07_nuit_1.edf,2TL07,1,control,1,1,,1,1,,,,,,,""[0,4]"",""[1,5]"",8000,,,,,,,"

lkorczowski commented 3 years ago

@RobinGuillard DONE

Please read log. When delta start is big, it may be an issue (truncated file ?)

Files processed: 
1AC26_cohort2.edf (sleep labels non uniform interval (count: [(7.0, 1), (22.0, 1), (23.0, 1), (30.0, 1147)]), taking median: 30.0
start time, file: 23:22:38 labels: 28/08/2020 23:22:38
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.37s) report... Bruxism(skipped) MEMA(report...done) DONE (16.99s)

1BA07_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1BA07_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1CB23_cohort2.edf (sleep labels non uniform interval (count: [(8.0, 1), (25.0, 1), (30.0, 1337), (3930.0, 1)]), taking median: 30.0
start time, file: 21:52:05 labels: 25/08/2020 21:52:05
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.13s) report... Bruxism(skipped) MEMA(report...done) DONE (16.64s)

1CC05_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1CC05_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1DA15_nuit_hab.edf (sleep labels non uniform interval (count: [(3.0, 1), (6.0, 1), (24.0, 1), (30.0, 981)]), taking median: 30.0
start time, file: 23:03:33 labels: 27/11/2019 23:03:57
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (3.44s) report... Bruxism(skipped) MEMA(report...done) DONE (14.40s)

1DA15_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1DL12_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1DL12_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1DM32_cohort2.edf (sleep labels non uniform interval (count: [(7.0, 1), (14.0, 1), (30.0, 1116)]), taking median: 30.0
start time, file: 00:40:16 labels: 27/08/2020 00:40:16
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (3.99s) report... Bruxism(skipped) MEMA(report...done) DONE (16.30s)

1GB18_nuit_hab.edf (sleep labels non uniform interval (count: [(3.0, 1), (4.0, 1), (30.0, 1060)]), taking median: 30.0
start time, file: 21:15:25 labels: 09/12/2019 21:15:26
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.90s) report... Bruxism(skipped) MEMA(report...done) DONE (18.09s)

1GB18_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1GF14_cohort2.edf (sleep labels non uniform interval (count: [(19.0, 1), (20.0, 1), (30.0, 1812)]), taking median: 30.0
start time, file: 18:51:41 labels: 26/08/2020 18:51:41
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.69s) report... Bruxism(skipped) MEMA(report...done) DONE (18.91s)

1GF14_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1GF14_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1HB20_nuit_1.edf (sleep labels non uniform interval (count: [(7.0, 1), (12.0, 1), (30.0, 812), (60.0, 2)]), taking median: 30.0
start time, file: 00:46:50 labels: 12/12/2019 00:46:53
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.21s) report... Bruxism(skipped) MEMA(report...done) DONE (14.07s)

1HB20_nuit_2.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(done) DONE (2.17s) report... Bruxism(skipped) MEMA(report...done) DONE (16.42s)

1HB20_nuit_2_resmed.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(done) DONE (2.37s) report... Bruxism(skipped) MEMA(report...done) DONE (18.50s)

1HS25_cohort2.edf (sleep labels non uniform interval (count: [(2.0, 1), (28.0, 1), (29.0, 1), (30.0, 1430)]), taking median: 30.0
start time, file: 22:08:31 labels: 25/08/2020 22:08:31
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (5.06s) report... Bruxism(skipped) MEMA(report...done) DONE (16.49s)

1MF19_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1MF19_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1MN09_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1MN09_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1NS30_cohort2.edf (sleep labels non uniform interval (count: [(18.0, 1), (28.0, 1), (30.0, 1213)]), taking median: 30.0
start time, file: 22:51:02 labels: 27/08/2020 22:51:02
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (4.21s) report... Bruxism(skipped) MEMA(report...done) DONE (16.37s)

1PA31_cohort2.edf (sleep labels non uniform interval (count: [(12.0, 1), (23.0, 1), (30.0, 1433)]), taking median: 30.0
start time, file: 22:12:07 labels: 25/08/2020 22:12:07
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (5.17s) report... Bruxism(skipped) MEMA(report...done) DONE (11.01s)

c'est 1PI07 hab :) ! 
1PI07_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1PI07_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1PT06_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1PT06_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1RA17_nuit_hab.edf (sleep labels non uniform interval (count: [(0.0, 1), (5.0, 1), (30.0, 875)]), taking median: 30.0
start time, file: 00:03:22 labels: 03/12/2019 00:03:25
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (3.02s) report... Bruxism(skipped) MEMA(report...done) DONE (16.60s)

1RA17_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1RF29_cohort2.edf (sleep labels non uniform interval (count: [(4.0, 1), (24.0, 1), (30.0, 1794)]), taking median: 30.0
start time, file: 19:56:56 labels: 26/08/2020 19:56:56
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.61s) report... Bruxism(skipped) MEMA(report...done) DONE (14.70s)

1SA14_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1SA14_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1SL21_nuit_2.edf (sleep labels non uniform interval (count: [(-86370.0, 1), (30.0, 847), (2312.0, 1), (5535.0, 1)]), taking median: 30.0
start time, file: 23:51:54 labels: 05/12/2019 22:19:15
delta_start 80841.0
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (22.55s) report... Bruxism(skipped) MEMA(report...done) DONE (19.95s)

1SL21_nuit_3.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(done) DONE (4.59s) report... Bruxism(skipped) MEMA(report...done) DONE (23.51s)

1UC22_nuit_1.edf (sleep labels non uniform interval (count: [(0.0, 1), (21.0, 1), (30.0, 1548)]), taking median: 30.0
start time, file: 19:28:39 labels: 08/12/2019 19:28:39
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.05s) report... Bruxism(skipped) MEMA(report...done) DONE (18.19s)

1UC22_nuit_2.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(done) DONE (3.51s) report... Bruxism(skipped) MEMA(report...done) DONE (20.42s)

1UC22_nuit_3.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1ZN04_nuit_hab.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

1ZN04_nuit_son.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

2CT04_nuit_1.edf (sleep labels non uniform interval (count: [(17.0, 1), (27.0, 1), (30.0, 840)]), taking median: 30.0
start time, file: 00:24:03 labels: 10/01/2020 00:24:03
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.63s) report... Bruxism(skipped) MEMA(report...done) DONE (16.23s)

2GA06_nuit_1.edf (sleep labels non uniform interval (count: [(16.0, 2), (30.0, 1003)]), taking median: 30.0
start time, file: 00:53:14 labels: 23/12/2019 00:53:14
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.62s) report... Bruxism(skipped) MEMA(report...done) DONE (17.59s)

2GR01_nuit_3.edf (sleep labels not found) preprocess... Bruxism(skipped) MEMA(done) DONE (2.76s) report... Bruxism(skipped) MEMA(report...done) DONE (16.15s)

2GR01_nuit_4.edf (sleep labels non uniform interval (count: [(15.0, 1), (22.0, 1), (30.0, 773)]), taking median: 30.0
start time, file: 01:27:08 labels: 25/11/2019 01:27:08
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.60s) report... Bruxism(skipped) MEMA(report...done) DONE (15.30s)

2LS05_nuit_1.edf (sleep labels non uniform interval (count: [(3.0, 2), (30.0, 843)]), taking median: 30.0
start time, file: 01:59:27 labels: 05/01/2020 01:59:27
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (2.91s) report... Bruxism(skipped) MEMA(report...done) DONE (16.19s)

2MJ02_nuit_1.edf (sleep labels non uniform interval (count: [(22.0, 1), (24.0, 1), (30.0, 710)]), taking median: 30.0
start time, file: 01:32:08 labels: 26/11/2019 01:32:08
, loaded) preprocess... Bruxism(skipped) MEMA(skipped) DONE (0.00s) report... Bruxism(skipped) MEMA(skipped) DONE (0.00s)

2SJ03_nuit_1.edf (sleep labels non uniform interval (count: [(19.0, 1), (26.0, 1), (30.0, 1121)]), taking median: 30.0
start time, file: 22:10:04 labels: 07/01/2020 22:10:04
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (3.70s) report... Bruxism(skipped) MEMA(report...done) DONE (20.86s)

3BS04_cohort2.edf (sleep labels non uniform interval (count: [(10.0, 1), (16.0, 1), (30.0, 1284)]), taking median: 30.0
start time, file: 23:13:44 labels: 26/08/2020 23:13:44
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (4.92s) report... Bruxism(skipped) MEMA(report...done) DONE (16.77s)

3CL02_cohort2.edf (sleep labels non uniform interval (count: [(6.0, 1), (24.0, 1), (26.0, 1), (30.0, 1679)]), taking median: 30.0
start time, file: 18:54:04 labels: 28/08/2020 18:54:04
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (6.35s) report... Bruxism(skipped) MEMA(report...done) DONE (26.58s)
3HS08_cohort2.edf (sleep labels non uniform interval (count: [(17.0, 1), (30.0, 1308), (2659.0, 1)]), taking median: 30.0
start time, file: 22:44:13 labels: 28/08/2020 22:44:13
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (5.56s) report... Bruxism(skipped) MEMA(report...done) DONE (18.04s)

3KB06_cohort2.edf (sleep labels non uniform interval (count: [(9.0, 1), (24.0, 1), (30.0, 1160)]), taking median: 30.0
start time, file: 23:15:36 labels: 27/08/2020 23:15:36
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (4.46s) report... Bruxism(skipped) MEMA(report...done) DONE (11.26s)

3NS01_cohort2.edf (sleep labels non uniform interval (count: [(13.0, 1), (22.0, 1), (30.0, 1425)]), taking median: 30.0
start time, file: 22:13:38 labels: 25/08/2020 22:13:38
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (5.15s) report... Bruxism(skipped) MEMA(report...done) DONE (16.57s)

3TR07_cohort2.edf (sleep labels non uniform interval (count: [(1.0, 1), (12.0, 1), (30.0, 1530)]), taking median: 30.0
start time, file: 20:24:18 labels: 27/08/2020 20:24:18
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (5.54s) report... Bruxism(skipped) MEMA(report...done) DONE (18.04s)

3TS03_cohort2.edf (sleep labels non uniform interval (count: [(11.0, 1), (15.0, 1), (19.0, 1), (30.0, 1339)]), taking median: 30.0
start time, file: 21:52:15 labels: 28/08/2020 21:52:15
, loaded) preprocess... Bruxism(skipped) MEMA(done) DONE (4.86s) report... Bruxism(skipped) MEMA(report...done) DONE (19.35s)

Reports created, process finished in 11.6 min
Results saved

lkorczowski / Tinnitus-n-Sleep

Sleep labels in report #103

This example was from Maxime's labels.

Any type is converted using read_sleep_file to

Codecov Report