bnosac / udpipe

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit
https://bnosac.github.io/udpipe/en
Mozilla Public License 2.0
209 stars 33 forks source link

Error in `[.data.table`(out, , `:=`(term_id, 1L:.N), by = list(doc_id)) #104

Closed swaheera closed 1 year ago

swaheera commented 1 year ago

I have the following data:

library(udpipe)
library(BTM)

my_data = structure(list(id = 1:8, reviews = c("I guess the employee decided to buy their lunch with my card my card hoping I wouldn't notice but since it took so long to run my car I want to head and check my bank account and sure enough they had bought food on my card that I did not receive leave. Had to demand for and for a refund because they acted like it was my fault and told me the charges are still pending even though they are for 2 different amounts.", 
                                            "I went to McDonald's and they charge me 50 for Big Mac when I only came with 49. The casher told me that I can't read correctly and told me to get glasses. I am file a report on your casher and now I'm mad.", 
                                            "I really think that if you can buy breakfast anytime then I should be able to get a cheeseburger anytime especially since I really don't care for breakfast food. I really like McDonald's food but I preferred tree lunch rather than breakfast. Thank you thank you thank you.", 
                                            "I guess the employee decided to buy their lunch with my card my card hoping I wouldn't notice but since it took so long to run my car I want to head and check my bank account and sure enough they had bought food on my card that I did not receive leave. Had to demand for and for a refund because they acted like it was my fault and told me the charges are still pending even though they are for 2 different amounts.", 
                                            "Never order McDonald's from Uber or Skip or any delivery service for that matter, most particularly one on Elgin Street and Rideau Street, they never get the order right. Workers at either of these locations don't know how to follow simple instructions. Don't waste your money at these two locations.", 
                                            "Employees left me out in the snow and wouldn’t answer the drive through. They locked the doors and it was freezing. I asked the employee a simple question and they were so stupid they answered a completely different question. Dumb employees and bad food.", 
                                            "McDonalds food was always so good but ever since they add new/more crispy chicken sandwiches it has come out bad. At first I thought oh they must haven't had a good day but every time I go there now it's always soggy, and has no flavor. They need to fix this!!!", 
                                            "I just ordered the new crispy chicken sandwich and I'm very disappointed. Not only did it taste horrible, but it was more bun than chicken. Not at all like the commercial shows. I hate sweet pickles and there were two slices on my sandwich. I wish I could add a photo to show the huge bun and tiny chicken."
)), class = "data.frame", row.names = c(NA, -8L))

I am trying to follow the instructions here https://cran.r-project.org/web/packages/BTM/readme/README.html to make the visualization.

I tried to begin this process by running the following code:


udpipe_download_model("english-ewt", model_dir = "~/Desktop/")

eng_model = udpipe_load_model("~/Desktop/english-ewt-ud-2.5-191206.udpipe")

# line with error
out = udpipe(my_data$reviews, object = eng_model)

But I now get this erorr:

Error in `[.data.table`(out, , `:=`(term_id, 1L:.N), by = list(doc_id)) : 
  Supplied 2 items to be assigned to group 1 of size 0 in column 'term_id'. The RHS length must either be 1 (single values are ok) or match the LHS length exactly. If you wish to 'recycle' the RHS please use rep() explicitly to make this intent clear to readers of your code.
In addition: Warning message:
In strsplit(x$conllu, "\n", fixed = TRUE) : input string 1 is invalid UTF-8

Does anyone know why this error is being produced and what can I do to fix it?

Thanks!


> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                    LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] BTM_0.3.6    udpipe_0.8.9

loaded via a namespace (and not attached):
[1] compiler_4.1.3    Matrix_1.4-0      tools_4.1.3       Rcpp_1.0.8.3      tinytex
jwijffels commented 1 year ago

Your text data needs to be provided in UTF-8 encoding. See the 'try-it-out' documentation at https://cran.r-project.org/web//packages/udpipe/vignettes/udpipe-tryitout.html or at https://bnosac.github.io/udpipe/docs/doc0.html

The 6th element is not in UTF-8 encoding (assuming your have your computer setup on a latin1 locale). The following probably works on your machine.

out = udpipe(iconv(my_data$reviews, from = 'latin1', to = 'UTF-8'), object = eng_model)