Closed KevKo1990 closed 2 years ago
Hey, thanks for the detailed issue and the example file! The issue is in your data, I think. I'm able to replicate your error, but it arises from the fact that there are empty lines in your input data. If you strip those, the classification works.
(remodnav) adina@muninn in ~/Downloads
❱ sed -i '/^\s*$/d' eye_tracking\(1\).txt
(remodnav) adina@muninn in ~/Downloads
❱ cat eye_tracking\(1\).txt
1054.7740745544434 567.3203587532043
1059.4937896728516 565.1391756534576
1087.8670692443848 587.2248977422714
1073.7462043762207 604.0173554420471
1066.3418197631836 582.63991355896
1139.8681640625 562.4450039863586
1140.2293395996094 563.3534359931946
1107.6242637634277 562.6706153154373
1109.509048461914 569.034161567688
1107.560634613037 569.4072031974792
1108.5669708251953 576.1746418476105
1101.1261940002441 587.4281072616577
1094.27490234375 573.6687612533569
1096.640281677246 578.154981136322
1096.1009788513184 591.5245485305786
1092.9481887817383 582.9961216449738
1092.528419494629 568.4891474246979
1083.4343147277832 577.3953473567963
1088.594913482666 579.4349098205566
1107.3037719726562 560.4691708087921
1102.074966430664 579.1347706317902
1107.9465866088867 570.4332447052002
1114.1997528076172 572.8529620170593
1099.4373321533203 585.4550743103027
1100.5931854248047 592.4616265296936
1101.7505264282227 581.5504002571106
1106.3683319091797 590.8732867240906
1126.8705368041992 604.4198155403137
1131.4482879638672 582.0984077453613
1131.4482879638672 582.0984077453613
1128.3570098876953 584.6828556060791
1127.5916290283203 579.1564965248108
1107.4720001220703 580.9134936332703
1110.9539794921875 579.1421735286713
1141.6521835327148 572.0102226734161
1151.3321685791016 577.0430660247803
1144.7561645507812 553.9843618869781
1134.9304389953613 551.4235389232635
1134.9708366394043 558.5087871551514
1134.2570686340332 562.5474536418915
1136.8513870239258 559.6298110485077
1132.6181602478027 558.2517457008362
1123.410472869873 572.329882979393
1132.7839851379395 568.1860792636871
1126.0827255249023 571.065194606781
1115.379695892334 564.9577403068542
1088.53609085083 552.6089519262314
1100.2128410339355 556.4097762107849
1122.6976203918457 560.4889011383057
1134.7531700134277 557.9530549049377
1059.6366119384766 557.4063992500305
1073.6090469360352 567.1656703948975
1091.6386985778809 601.1293888092041
1087.3554611206055 576.4352238178253
1084.1843605041504 572.0255756378174
1082.4741554260254 573.7123095989227
1058.2696723937988 576.0424518585205
1055.9429168701172 579.5147001743317
1051.8917655944824 585.9971272945404
1048.7213516235352 586.3098192214966
1039.9726867675781 592.834507226944
1033.1046867370605 593.3007287979126
1045.0513458251953 588.118861913681
1054.562759399414 578.6194324493408
1053.5430335998535 584.6184504032135
1054.3145370483398 593.9500272274017
1043.9703941345215 583.5110414028168
1044.0279579162598 582.5770854949951
1048.0718421936035 605.4385828971863
1043.2265281677246 598.452855348587
1060.1206970214844 565.0927948951721
1022.0997619628906 541.0411047935486
1027.0194625854492 585.5226016044617
1042.053508758545 596.5517270565033
1108.2175827026367 620.2361047267914
1120.1012992858887 606.204879283905
1127.1065139770508 574.5857548713684
1149.7362899780273 612.8430247306824
1115.5833435058594 620.0620079040527
1106.9770431518555 629.5080399513245
1111.227035522461 575.0091683864594
1111.2773323059082 564.1863262653351
1108.073787689209 573.1479513645172
1111.2110137939453 576.2617385387421
1109.5995712280273 568.159943819046
1108.6676788330078 564.9553263187408
1152.6319313049316 531.3502138853073
1282.9631423950195 415.2369511127472
1298.6566543579102 313.3840173482895
1272.1385192871094 238.0040037631988
1278.7775802612305 249.966521859169
1284.5343589782715 250.20884603261945
1285.8380699157715 246.68699830770493
1284.1532135009766 245.5621039867401
1295.6406784057617 227.2441506385803
1299.1432571411133 233.3820855617523
1289.5955085754395 247.2242876887321
1286.7933082580566 247.20054209232327
1291.8848419189453 249.48531746864316
1272.5455856323242 231.964253783226
1283.1925964355469 220.6830114126205
1291.179141998291 211.39079868793482
1292.134609222412 210.70489615201944
1277.74658203125 245.74874550104136
1217.1868515014648 336.2017813324928
1308.2143592834473 221.70179486274716
1308.2143592834473 221.70179486274716
1332.7217102050781 247.4624919891357
1330.345687866211 230.49596965312955
1334.5793724060059 200.69163322448728
1322.168197631836 221.88373714685437
1318.458137512207 230.0630450248718
1333.494701385498 208.80337357521051
1326.2238693237305 209.80336815118784
1329.287166595459 213.01373898983
1355.6187057495117 188.29972296953193
1355.0450134277344 218.55484753847117
1350.5336952209473 209.42072689533234
1350.8926391601562 206.39026254415504
1345.6965065002441 219.85406398773193
1346.6005325317383 210.4570841789245
1349.4575500488281 197.01427638530726
1349.5458984375 201.53403460979456
1336.1417770385742 211.07163727283475
1344.9931526184082 212.3477035760879
1340.3471374511719 219.95983690023417
1333.0910110473633 220.2659466862678
1344.2718887329102 199.6949860453605
1339.6259880065918 206.9429531693458
1335.2061080932617 232.93816924095148
1338.566722869873 208.6932152509689
1337.8807640075684 200.4255956411361
1335.8140754699707 202.33742594718927
1330.9332847595215 204.9282386898994
1328.0816459655762 217.69140422344205
1323.5829734802246 223.9583507180214
1326.936149597168 206.06628924608225
1331.8769073486328 208.5886895656585
1331.5080642700195 218.46721976995462
1342.8633499145508 219.56320255994797
1343.5133743286133 248.5138964653015
1291.6427993774414 516.9105255603789
448.0543041229248 467.4430167675018
448.0543041229248 467.4430167675018
1360.8380126953125 751.3775861263275
1360.8380126953125 751.3775861263275
1278.369369506836 756.1146676540375
1272.1778297424316 756.5988492965698
1255.6648063659668 739.158525466919
1255.2773094177246 744.6975982189178
1242.6923561096191 738.5959053039551
1240.4677391052246 746.4400148391724
1239.8784828186035 812.734147310257
1246.868019104004 758.3018374443054
1243.1351280212402 753.4046924114227
1269.7058486938477 716.8673086166382
1254.071159362793 746.2193119525909
1252.5091552734375 740.8710408210754
1252.906608581543 743.0390274524689
1256.7271614074707 744.0558314323425
(remodnav) adina@muninn in ~/Downloads
❱ remodnav eye_tracking\(1\).txt out.tsv 0.0181155685774209 60 --savgol-length 0.05 --median-filter-length 0.05 --velthresh-startvelocity 300 --pursuit-velthresh 5 --saccade-context-window-length 2 --dilate-nan 0.02 --min-saccade-duration 0.02
WARNING:Computed velocity exceeds threshold. Inappropriate filter setup? [1039.0 > 1000.0 deg/s]
Please reopen if the issue persists, and obviously feel free to open new issues if something else is wrong. :)
Thank you for looking into it, @adswa! There are indeed empty spaces, however, the missing data is to some extent planned. In our case, we had 3 monitors in place but only the central one was recorded with a eye tracker. If people watched on the other screens (or blinked), nan (empty data points) were created.
Does Remodnav currently not handle missing data points? What is your recommendation on how to treat them?
Hey, thanks - you're right. The failure actually shouldn't arise. I took a closer look into your data, its a bit of an unfortunate edge case we never hit because we've not dealt with such low sampling rates. The crash seems to happen because of an unfortunate combination of a missing value followed by several identical coordinates. This causes a high jump in velocities followed by interpolation to the previous (nan) velocity value, and the median filter dealt poorly with it due to identical values that followed. In the end, an event with only nan velocity ended up in the classification, and then numpy's np.max()
crashed. I pushed a basic safeguard in #36, but its not really a pretty one, because due to the nature of the data, certain properties end up not being computed for this sample (e.g., amplitude or peak velocity).
I could get around this by removing one of the duplicated lines in the input data after the last nan, or by removing the last nan, but that, too, isn't really a fix. Does this happen for much of your data, or is it a rare case?
I have released 1.1.1
with the safeguard to PyPi.
@adswa I tested it and it fixed the errors on my end with a 60Hz dataset. Thank you!
Hi,
thank you @adswa for pushing version 1.1 to pip. Unfortunetaly, my error from https://github.com/psychoinformatics-de/remodnav/issues/28 persists. I checked if the new error warning occur, however, this is not the case. My variable selection causes no warning messgage.
I use the following parameter settings to run the algorithm (I optimized the settings based on a self-labeled dataset):
To replicate the error, I attached the crashing part of my dataset: eye_tracking.txt
Do you have any idea why with my data the algorithm crashes? Thank you already for your help!