Open Iman-Idrees opened 2 years ago
Try suggestions reported here: https://github.com/dmirman/gazer/issues/3
Hi again,
I was able to load in the edfs following your guidance on the previous posts. Thank you! However, I am now having an issue with the find_messages_edf function where I get the following error:
find_messages_edf(file_list=file_list_edf, varnames=c("TRIALID", "RT", "ACCURACY", "RESPONSE"), patterns=c("TRIALID", "RT", "ACCURACY", "RESPONSE"), output_dir)
Reading Messages from 011.edf ...Success! (52785 messages)
Error: Argument 2 must be length 150, not 300
Any advice on how to fix this?
Thanks in advance, Iman
I have no clue what you named your variables. Can you send me one of your EDF files?
Sure, I've attached one here.
This should do the trick:
find_messages_edf(file.choose(), varnames=c("TRIALID", "!V TRIAL_VAR RT","!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), patterns=c("TRIALID", "!V TRIAL_VAR RT", "!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), output_dir=output)
You have a lot of messages so the patterns need to be pretty specific or you end up picking up more rows than you want.
Thanks, I tried the code suggested and it returned the following error message:
Error in file(file, ifelse(append, "a", "w")) :
cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
Error in file(file, ifelse(append, "a", "w")) :
cannot open the connection
file.choose() should be replaced with an object that has each of your EDF files.
Can you also show me your code?
Sure...
directory_edf= "/root/capsule/code/R/raw_data"
directory_csv_from_edf_conversion ="/root/capsule/code/R/raw_data"
file_list_edf <- list.files(path=directory_edf, pattern= ".edf")
parse_edf(file_list = file_list_edf, directory_csv_from_edf_conversion, type = "pupil")
file_list_pupil_samp <- list.files(path=directory_csv_from_edf_conversion, pattern = ".csv")
pd <- merge_gazer_files(file_list_pupil_samp, filetype = "edf")
find_messages_edf(file_list =file_list_edf, varnames=c("TRIALID", "!V TRIAL_VAR RT","!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), patterns=c("TRIALID", "!V TRIAL_VAR RT", "!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), output_dir= "/root/capsule/code/R/raw_data")
This seems to work but I receive the following error as before:
Error: Argument 2 must be length 150, not 149
Having a look at the output it seems that the trials start from 0 instead of 1 which I think might be the issue here? Is there a way to change this to begin at 1?
SR records the first trial as 0–you can easily change that by adding 1 to the trial id. That is not the issue. Your issue seems to be a wonky edf file. One of your files is missing a row of data. I would go through each of your edf files and see which one only has 149 rows of data.
Get Outlook for iOShttps://aka.ms/o0ukef
From: Iman-Idrees @.> Sent: Wednesday, August 18, 2021 4:43:46 PM To: dmirman/gazer @.> Cc: Geller, Jason @.>; Comment @.> Subject: [External] Re: [dmirman/gazer] Loading edfs on Code Ocean (#19)
Sure...
directory_edf= "/root/capsule/code/R/raw_data"
directory_csv_from_edf_conversion ="/root/capsule/code/R/raw_data"
file_list_edf <- list.files(path=directory_edf, pattern= ".edf")
parse_edf(file_list = file_list_edf, directory_csv_from_edf_conversion, type = "pupil")
file_list_pupil_samp <- list.files(path=directory_csv_from_edf_conversion, pattern = ".csv")
pd <- merge_gazer_files(file_list_pupil_samp, filetype = "edf")
find_messages_edf(file_list =file_list_edf, varnames=c("TRIALID", "!V TRIAL_VAR RT","!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), patterns=c("TRIALID", "!V TRIAL_VAR RT", "!V TRIAL_VAR ACCURACY", "!V TRIAL_VAR RESPONSE"), output_dir= "/root/capsule/code/R/raw_data")
This seems to work but I receive the following error as before: Error: Argument 2 must be length 150, not 149
Having a look at the output it seems that the trials start from 0 instead of 1 which I think might be the issue here? Is there a way to change this to begin at 1?
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dmirman/gazer/issues/19#issuecomment-901417554, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEMTQEDDG2XNMQ4OLBKZKWLT5QLQFANCNFSM5BGZAYMQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.
Hi, I'm having an issue with uploading my EDF file through Code Ocean, when using parse_edf. I get the following error: Error in names(output$header) <- c("eyetrial", "starttime", "endtime", : 'names' attribute [4] must be the same length as the vector [1]
Here is my EDF file: https://drive.google.com/file/d/1yx-35FRNdNQ5SPUKBoJLgDnc6qU-VAD9/view?usp=sharing
Please let me know if there is a way to resolve this issue.
With thanks,
Hi, I'm having an issue with uploading my EDF file through Code Ocean, when using parse_edf. I get the following error: Error in names(output$header) <- c("eyetrial", "starttime", "endtime", : 'names' attribute [4] must be the same length as the vector [1]
Here is my EDF file: https://drive.google.com/file/d/1yx-35FRNdNQ5SPUKBoJLgDnc6qU-VAD9/view?usp=sharing
Please let me know if there is a way to resolve this issue.
With thanks,
Hi. There appears to be something wrong with the EDF file you are trying to use. With the edfR function contained within parse_edf
it is trying to find a column name for trial and it cannot (which is why the error is being thrown). How did you collect this collect data?
Hello,
I am getting the same error as @skalhan14. Any updates on how it was resolved?
I collected my data using SR EyeLink 1000 Plus using PsychoPy experimental design. My design was based on one long continuous trial. So when I export the sample report using DataViewer, I get a column for trial ID that says '1' for all data.
Any help is appreciated. Thanks!
Hi @skalhan14,
edfR (which is not no longer being maintained) is trying to parse trial information through edf.trials.
If you have access to DataViewer, just export the files as csvs or txt files and use gazeR package to process those.
Thanks @jgeller112!
Are there any specific columns that have to be in the sample report? Or can it include any variables I need? I am trying to analyze pupil size timestamped to specific messages only.
[image0.jpeg] Min these Jason Geller, Ph.D.(he/him/his) Lecturer Princeton University Website: www.drjasongeller.com
On Feb 22, 2023, at 6:00 PM, sarahakhras @.***> wrote:
Are there any specific columns that have to be in the sample report? Or can it include any variables I need? I am trying to analyze pupil size timestamped to specific messages only.
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdmirman%2Fgazer%2Fissues%2F19%23issuecomment-1440967227&data=05%7C01%7Cjg9120%40princeton.edu%7C20a6019d8b854d42139c08db1528a12d%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127036481019921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GOsgF2vnSEg5zuiiZ3g9Dvtvbo6weznuAZccnmvR89M%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMTQEDET5OB7IXFMLRTVDLWY2LBZANCNFSM5BGZAYMQ&data=05%7C01%7Cjg9120%40princeton.edu%7C20a6019d8b854d42139c08db1528a12d%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127036481019921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kwdRSwZjRt%2B5JHy1tUIa4qAnDn21RlFWvXPn0aLoh9s%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
As long as you have messages it should work!
Jason Geller, Ph.D.(he/him/his) Lecturer Princeton University Website: www.drjasongeller.com
On Feb 22, 2023, at 6:00 PM, sarahakhras @.***> wrote:
Are there any specific columns that have to be in the sample report? Or can it include any variables I need? I am trying to analyze pupil size timestamped to specific messages only.
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdmirman%2Fgazer%2Fissues%2F19%23issuecomment-1440967227&data=05%7C01%7Cjg9120%40princeton.edu%7C20a6019d8b854d42139c08db1528a12d%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127036481019921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GOsgF2vnSEg5zuiiZ3g9Dvtvbo6weznuAZccnmvR89M%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMTQEDET5OB7IXFMLRTVDLWY2LBZANCNFSM5BGZAYMQ&data=05%7C01%7Cjg9120%40princeton.edu%7C20a6019d8b854d42139c08db1528a12d%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127036481019921%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kwdRSwZjRt%2B5JHy1tUIa4qAnDn21RlFWvXPn0aLoh9s%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
Thanks!
I am trying to read the .csv file now, but I'm getting the following error:
Error in data.table::fread(pupil_path) : Input is empty or only contains BOM or terminal control characters
Here's my file: test.csv
I'm not sure what the problem is. I appreciate your help!
There must be something weird going on with your csv files
Get Outlook for iOShttps://aka.ms/o0ukef
From: sarahakhras @.> Sent: Wednesday, February 22, 2023 6:38:13 PM To: dmirman/gazer @.> Cc: Jason Geller @.>; Mention @.> Subject: Re: [dmirman/gazer] Loading edfs on Code Ocean (#19)
Thanks!
I am trying to read the .csv file now, but I'm getting the following error:
Error in data.table::fread(pupil_path) : Input is empty or only contains BOM or terminal control characters
I'm not sure what the problem is. I appreciate your help!
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdmirman%2Fgazer%2Fissues%2F19%23issuecomment-1441006019&data=05%7C01%7Cjg9120%40princeton.edu%7C069d3c5ad91949876b9608db152dddb6%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127058960394569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=skMQY%2FLWi7SlBEYAwzARQ5RA3s48WEhkJkQ55SN1mJA%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMTQEB6M4MSA72IRBBICB3WY2POLANCNFSM5BGZAYMQ&data=05%7C01%7Cjg9120%40princeton.edu%7C069d3c5ad91949876b9608db152dddb6%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127058960394569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7rKTaex6t47lNr5wDHzHTLK%2BndG5ksvNX7P7gdNz0PY%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
Can you just send me one of your files?
Get Outlook for iOShttps://aka.ms/o0ukef
From: Jason Geller @.> Sent: Wednesday, February 22, 2023 6:39:25 PM To: dmirman/gazer @.>; dmirman/gazer @.> Cc: Mention @.> Subject: Re: [dmirman/gazer] Loading edfs on Code Ocean (#19)
There must be something weird going on with your csv files
Get Outlook for iOShttps://aka.ms/o0ukef
From: sarahakhras @.> Sent: Wednesday, February 22, 2023 6:38:13 PM To: dmirman/gazer @.> Cc: Jason Geller @.>; Mention @.> Subject: Re: [dmirman/gazer] Loading edfs on Code Ocean (#19)
Thanks!
I am trying to read the .csv file now, but I'm getting the following error:
Error in data.table::fread(pupil_path) : Input is empty or only contains BOM or terminal control characters
I'm not sure what the problem is. I appreciate your help!
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdmirman%2Fgazer%2Fissues%2F19%23issuecomment-1441006019&data=05%7C01%7Cjg9120%40princeton.edu%7C069d3c5ad91949876b9608db152dddb6%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127058960394569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=skMQY%2FLWi7SlBEYAwzARQ5RA3s48WEhkJkQ55SN1mJA%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMTQEB6M4MSA72IRBBICB3WY2POLANCNFSM5BGZAYMQ&data=05%7C01%7Cjg9120%40princeton.edu%7C069d3c5ad91949876b9608db152dddb6%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127058960394569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7rKTaex6t47lNr5wDHzHTLK%2BndG5ksvNX7P7gdNz0PY%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
Thanks for your help!
Here's the edf file: https://drive.google.com/file/d/1Y81OzAibkLARiYyJffJWssukiy9cP89j/view?usp=sharing.
The link is not to a csv file. Can you send that?
Here's the csv file: test.csv
Okay. The csv file does not contain the columns needed to use gazeR.
you need at the min these columns from DataViewer:
RECORDING_SESSION_LABEL
TRIAL_INDEX
AVERAGE_IN_BLINK RIGHT_IN_BLINK or LEFT_IN_BLINK
AVERAGE_PUPIL_SIZE RIGHT_PUPIL_SIZE LEFT_PUPIL_SIZE
IP_START_TIME
SAMPLE_MESSAGE
I'm still getting the same error even after including all the required columns. Here's my updated csv file if you'd like to take a look at it: testest.csv
If you are using merge_gazer_files
It is expecting an xslx file. You can change the csv to xslx or copy the below code.
#' Merge eye files from a ufolder
#' if filetype is from sr renames column variables and makes them lower case
#' Places time in ms
#' if edf files just reads them in
#'@import tidyverse
#'@import vroom
#' @param file_list path to .xls files
#' @param blink_colname name of your blink colname:AVERAGE_IN_BLINK, LEFT_, #RIGHT
#' @param pupil_colname name of your pupil colname:AVERAGE_IN_Pupil, LEFT_, #RIGHT
#' @param filetype if reading raw edf file use 'edf' if using sr files use type 'sr'
#' @export
#' @return data frame containing pupil data
merge_gazer_files <- function (file_list, blink_colname="blink", pupil_colname="pupil", filetype="sr") {
#file list is path to .xls files
#vroom is faster
library(data.table)
if (filetype=="sr") {
dataset <- do.call("rbind", lapply(file_list, FUN=function(files){
for (i in 1:length(files)){
if(file.exists(files[i])){
message( "now processing:", files[i])
}
}
fread(files, header=TRUE, na.strings = ".", fill=TRUE)})) #vroom makes reading in files quick
change_name <- dataset %>%
dplyr::select(!!quo_name("subject"):= RECORDING_SESSION_LABEL,
!!quo_name("blink"):= blink_colname,
!!quo_name("pupil"):= pupil_colname,
!!quo_name("trial"):= TRIAL_INDEX,
!!quo_name("message"):= SAMPLE_MESSAGE,
everything())
names(change_name) <- tolower(names(change_name))
change_name$time <- change_name$timestamp-change_name$ip_start_time
return(as_tibble(change_name))
}
if (filetype=="edf") {
dataset <- do.call("rbind", lapply(file_list, FUN=function(files){
for (i in 1:length(files)){
if(file.exists(files[i])){
message( "now processing:", files[i])
}
}
fread(files, header=TRUE, na.strings = "NA", fill=TRUE)})) #vroom makes reading in files quicke
return(as_tibble(dataset))
}
}
x=merge_gazer_files(file, blink_colname = "RIGHT_IN_BLINK", pupil_colname = "RIGHT_PUPIL_SIZE")
Thanks for your reply!
I am not using merger function. I'm only trying to see if it can read my data. Here's my code:
setwd(here())
csv.directory=here()
file_list <- list.files(path=edf.directory, pattern=".csv")
pupil_path <- system.file("extdata", "testest.csv", package = "gazer")
pupil_raw <- data.table::fread(pupil_path)
pupil_data <- as_tibble(pupil_raw)
head(pupil_data)
I thought I don't need merger function since I'm not using edf files. Is there anything missing from my code?
Are you doing this in Code Ocean. If so, just run it from local machine.
Also the system file thing is not needed (not sure what that does as that’s specific to the gazeR stuff). You can Run this:
Fread(file.choose()) # input the file
Jason Geller, Ph.D.(he/him/his) Lecturer Princeton University Website: www.drjasongeller.com
On Feb 23, 2023, at 5:16 PM, sarahakhras @.***> wrote:
Thanks for your reply!
I am not using merger function. I'm only trying to see if it can read my data. Here's my code:
setwd(here()) csv.directory=here() file_list <- list.files(path=edf.directory, pattern=".csv")
pupil_path <- system.file("extdata", "testest.csv", package = "gazer") pupil_raw <- data.table::fread(pupil_path)
pupil_data <- as_tibble(pupil_raw)
head(pupil_data)
I thought I don't need merger function since I'm not using edf files. Is there anything missing from my code?
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdmirman%2Fgazer%2Fissues%2F19%23issuecomment-1442501686&data=05%7C01%7Cjg9120%40princeton.edu%7C1f3dbc4a1ab04ae8a03508db15eb96c3%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127873808673603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=z7TBIxV9I5YxpdOV9V14tyjJTFeIMUtzzmBK8j54erQ%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMTQEGEI6GLD3MGLW65MELWY7OTFANCNFSM5BGZAYMQ&data=05%7C01%7Cjg9120%40princeton.edu%7C1f3dbc4a1ab04ae8a03508db15eb96c3%7C2ff601167431425db5af077d7791bda4%7C0%7C0%7C638127873808673603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2F8McLHBx9%2Bt5TD6Vdvvwubzdk6Cdpp3XYZ7AUntnaJA%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
Hi,
I would like to use gazeR to analyse my pupil data. I tried to load in an edf on Code Ocean but I received the following error message:
I've also tried to load in a few different edfs but I receive the same error message each time. Any advice on what the issue might be here would be greatly appreciated!
Many thanks, Iman