Renderers (other than TSV) disappear, when 'subtotals = FALSE' AND 'tsv = TRUE'

[ghost]

    rpivotTable(mtcars, rows = "am", cols = "cyl", width = "90%", height = "40%", subtotals = FALSE, tsv = FALSE)  # OK
    rpivotTable(mtcars, rows = "am", cols = "cyl", width = "90%", height = "40%", subtotals = TRUE, tsv = TRUE)    # OK
    rpivotTable(mtcars, rows = "am", cols = "cyl", width = "90%", height = "40%", subtotals = TRUE, tsv = FALSE)   # OK
    rpivotTable(mtcars, rows = "am", cols = "cyl", width = "90%", height = "40%", subtotals = FALSE, tsv = TRUE)   # NOT OK!

Expected behavior In the 4th case, the list of available renderers should be like this.

Instead, only "TSV Export" is shown as available.

I'm hoping it's just a minor fix.

[ghost]

In rpivottable.js I attempted the following modifications, and they seem to work.

Also, by adding the $.pivotUtilities.renderers (first red line on the right panel), I got all the renderers! Do you remember this discussion?

Anyway, these are merely quick tests. The ability to select the best course of action rests with you.

P.S. It is possible that the giveItBack function will need to be modified. At this time, I have not detected any issues. I have done these modifications on the version I have locally, but it would be nice if that problem didn't exist on your end. (Have I been so wrong again? And yet, I also try to do it all the right way.)

[ghost]

Okay, since the aforementioned issue is being addressed in some way, I'll use this chance to ask about a more difficult issue for which I haven't been able to come up with a solution.

When I tried to switch from R 4.1.2 to 4.2.1 in Windows 10 (I tried both with package update and without) my rmarkdown reports (all encoded in UTF-8) don't work because queries containing Greek characters are not sent correctly to the database (a Sybase ASE 15, with CP1253 encoding). I connect to the db using a command like this dbConnect(odbc::odbc(), "PIND_64", encoding = "CP1253"), where "PIND_64" is a system ODBC that uses the 64bit ASE driver.

The problem is not related to RStudio because I tried to rmarkdown::render the .rmd file from RGUI as well. The report is generated correctly by the 4.1.2 RGUI but fails by the 4.2.1 one.

Unfortunately I can't make a reproducible example of this problem, so I can post a S.O. question about it. Also, I failed to find anything useful by googling.

Oh! One additional problem that occurred: Within the rmd reports, I often source a file of type .r (which contains generic functions and also another query with Greek characters). Up to version 4.1.2, in order for files like this (*.r) to work correctly, they had to be saved with native encoding (CP1253). (At least that was the conclusion I had come to). With R 4.2.1 they are not even read correctly via source.

As you understand, I'm simply not even considering switching to 4.2.# under these circumstances.

I apologize if I bored you. I would greatly appreciate any ideas you may have.

[fraupflaume]

I've pushed the updates to rpivotTable.js so that's fixed. Sorry about that.

BLUF or TL;DR;

As far as the Greek, RStudio, and R... I've been playing around with this a little bit. I haven't gone so far as to create an arbitrary database, but I did play around with RMD inline Greek, Greek strings, and importing Greek from a source file.

I'm not having these issues. So, my first question is whether or not I've truly replicated any of the things that are causing problems on your end.

If I have, I'm not sure if this is an encoding issue, a Windows issue, or if there is something else going on. I can tell you that the launch of 4.2 was supposed to fix some very long-awaited issues between base R and UTF-8—specifically to address issues regarding language encoding (languages of people, not programming).

And all the rest...

My current versions:

What	Version
OS	x86_64-apple-darwin17.0 (latest Mac OS)
R	4.2.1
RStudio	2022.07.2+576 "Spotted Wakerobin"
R Markdown	2.18
knitr	1.40

I created two small script files with the exact same content. I saved one with CP1253 encoding. I saved the other with ISO8859-7 encoding. I don't know exactly what type of content you have, so I just used Greek strings in a data frame and in a graph. Here's the code I used:

library(tidyverse)

# create a data frame
x <- c("Κόκκινο", "Κίτρινο"," Πράσινο")
y <- c(5,4,9)
df <- data.frame(x, y)

ggplot(df, aes(x, y)) + geom_point()

When I source the file, it renders correctly (both encodings).

After closing and reopening it RStudio, all I see is '?' in place of the Greek letters. However, when I sourced it in RMD (RMD is NOT encoded in anything other than UTF-8), It rendered as I had expected. (By the way, I have no idea what those words mean...I copied it from the net.) I initially encoded the RMD as Greek, but nothing worked.

Here's what I've got in the RMD.

---
title: "Untitled"
author: "me"
date: "2022-11-12"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = F)
# knitr::read_chunk("grkScript.R")
# Sys.setlocale("LC_CTYPE", "en_US.UTF-8")
library(tidyverse)  # here so that it doesn't echo when source is called
library(gridExtra)
library(rpivotTable)

data(mtcars)

$('head').append('<link rel="preconnect" href="https://fonts.googleapis.com"><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin><link href="https://fonts.googleapis.com/css2?family=Noto+Sans&family=Noto+Sans+Mono&family=Noto+Serif&display=swap" rel="stylesheet">')

this <- source("grkScript.R", encoding = "ISO8859-7")

that <- source("grkScript2.R", encoding = "CP1253")
# show the plots
grid.arrange(this[[1]], that[[1]], nrow = 1)

Όλα αυτά είναι ωραία και κομψά. Δεν ξέρω καθόλου ελληνικά, οπότε χρησιμοποιώ το Google Translate.

tellMe <- " Όλα αυτά είναι ωραία και κομψά. Δεν ξέρω καθόλου ελληνικά, οπότε χρησιμοποιώ το Google Translate."

tellMe

![image](https://user-images.githubusercontent.com/32872512/201495711-10b35514-2fb6-4bd8-a332-4f9772783757.png)

[ghost]

Another invitation ?!? At least with this one, I have the option of declining :) (Thanks for making me laugh. I needed it. Why did you have to be so nice?) You know, I really hope I didn't have to even open Rstudio again, much less deal with a language I don't know. (Speaking of languages, learning German right now is what I find most fascinating.) All you might need is a better way to be notified of new messages. 😁

---

On the laptop I have at home (R 4.2.1), the behavior is as you describe. No problem. Perhaps the issue is limited in the database communication. I'll do some more careful testing at work, starting Monday, and I will post here my findings (if you can think of a better location, please let me know).

Herzlichen Dank.

[ghost]

I think you have already solved the source problem. I wasn't passing the encoding argument to the source command 🤦‍♂️. As a result it used the native.enc, which is Greek_Greece.utf8 (and, in R 4.1.# was Greek_Greece.1253). Sheesh! I know that the time has come to give up programming. I am close, I swear.

[fraupflaume]

I pretty much learned German from my grandparents speaking it around me. I studied Russian for years in college. With no one to use it with and no when to correct it when I screw up...yea, I don't remember any of it now.

I have a question...so in English (and probably in many other languages) we primarily use Greek letters for specific elements in math, like $\theta$ for angles or $\Delta$ for the difference or change. What do Greeks use? I feel like it just wouldn't have the same significance. (Like how it just sticks out in English.)

As far as messages...ya, I kind of suck at that. It's not just messages. Really, it's communication in general. I'm a work in progress...for like my entire life.

I think it's fine to put your server/encoding content here. I doubt all that many people are looking at this repo!

By the way, (you may already know this) I read that CP1253 violates a few issues with encoding, which is why ISO8859-7 differs. I'm not sure how big of a deal it is. I just thought I would mention it.

I downloaded a trial of SAP ASE to see if I could help....the first hurdle—it doesn't run on Mac; I use Virtual Box for cybersecurity reasons with several different Linux OS. I don't have one with Windows, though. It's been a rather frustrating journey so far! When I finally got the server running I couldn't interact with it (operator error). I've finally got it running...my next hurdle... I need a Greek-encoded database. I'm working on translating one of the example databases. I'm having all sorts of fun trying to regex the strings out of the bash file that creates that example database. The Greek will probably be laughably poor grammar, but meh, it'll at least get me to the point where I might be able to reproduce some of the issues you're having.

[ghost]

Mathematical language is the same for everyone, it could not be otherwise. Greeks use the same mathematical symbols as everybody else. It's just that these symbols are a little more familiar to us.

The truth is that your unique relationship with communication can lead to misunderstandings such as: "No response is a response", especially if one has a good reason to worry about this.

If I continue to have the honor and pleasure of communicating with you, I will soon begin writing to you in German. 😊

I also have a couple of questions.

1. I noticed that you didn't add pivotUtilities.renderers in if(x.subtotals). Was there a problem, or did you not like the idea of having all the renderers available?
2. How can you find out that the encoding of a file is, say "ISO8859-7" (as in "grkScript.R") if you dont't know it?

I don't think you need to go to that much trouble (database installations). Something has changed in the way the query is sent to the database. I wonder if it is a problem with the odbc package. In the meantime I found this related issue. The last comments are relatively recent. This one worked for me, but in reverse. Here is an example of how I had to use iconv in order to get the correct results.

crs.odbc <- dbConnect(odbc::odbc(), "PIND_64", encoding = "CP1253")

dbGetQuery(crs.odbc, iconv("
  select *
  from my_tbl
  where ta_type in ('Άνω Διάβαση Οδού','Οδική Γέφυρα')
","CP1253", "UTF-8"))

But even if I were willing to convert all my queries this way (which I'm not), I would still have the unsolved problem of queries inside {sql} chunks, for which I have found no way to send them correctly (actually, they only function well when I am working interactively in Rstudio ver. 1.4.1717 but not when I am knitting. In Rstudio 2022.07.2 they fail both ways).

Another report that connects to a different (SQL Server) database succeeds in knitting, but all of the Greek strings that are returned from the database are incomprehensible. I should also add that, in this case, none of the queries contained Greek characters, so I can't say for sure that we wouldn't have had the above problem here as well. 🤢 - In short: literally nothing works correctly when database communication is required (and all tests have been done on 2 computers). And I really can't believe that for so many years everything worked by accident. I'm beginning to feel bad for putting you in this situation. Stop wasting your time on this. Just think about my first question above, and close this issue.

Just for the record. R 4.1.2

l10n_info()
#> $MBCS
#> [1] FALSE
#> 
#> $`UTF-8`
#> [1] FALSE
#> 
#> $`Latin-1`
#> [1] FALSE
#> 
#> $codepage
#> [1] 1253
#> 
#> $system.codepage
#> [1] 1253

R 4.2.2

l10n_info()
#> $MBCS
#> [1] TRUE
#> 
#> $`UTF-8`
#> [1] TRUE
#> 
#> $`Latin-1`
#> [1] FALSE
#> 
#> $codepage
#> [1] 65001
#> 
#> $system.codepage
#> [1] 65001

P.S. To keep myself motivated, I've set the aim of learning gt. This ebook is brand new. I thought you might be interested.

[fraupflaume]

Klingt nach einer tollen Idee, schreib auf Deutsch!

1)--- Warum nicht? When I tested it with$.pivotUtilities.renderers attached to if(subtotals), rpivotTable will not render if subtotal = T. Were you able to get a different outcome? (Am I missing something!!!????!)

2) I used Google. However, I can say that there are many resources out there, but some aren't great. This one works well, but you have to know to add "CP" to the Windows codes.

As far as the incomprehensible strings--- can you tell me if, without knitting or rendering, you just printed the data frame (or whatever structure) to the console, does it render correctly? Is it just on the graph that reflects nonsense?

If it's nonsense regardless, have you tried using stringi::stri_encode instead of iconv? Also, it may pay to try leaving the encoding off in dbConnect. Right now it looks as if you're trying to change the encoding 2 times... Since you've been at this for a while, my ideas may be 'been there-done that.'

gt is a pretty snazzy package! I haven't seen that resource before. It's on the list 'useful' now.

[ghost]

Ich mag Geheimnisse aber nicht so sehr... (This post is for pivottable only. I will post later for the rest) If it doesn't work for you, you did well to skip it !

[ghost]

On the SQL Server rmd report case.

The data.frame does NOT print correctly to the console. (I see the same characters as in the plot). If I omit the encoding = "CP1253" from dbConnect, the strings seem to be correct indeed, but they aren't. Check lines 5 and 27 in "nodes" (in SQL_Server_sankey_3.zip)

(I attach three pairs of "links" and "nodes" files, one for each case).

1. SQL_Server_sankey_1.zip: R 4.1.2, (connection with encoding ON in dbConnect).
2. SQL_Server_sankey_2.zip: R 4.2.2, (connection with encoding ON in dbConnect).
3. SQL_Server_sankey_3.zip: R 4.2.2, (connection with encoding OFF in dbConnect).

That's the error in case 3.

library(networkD3)
load("SQL_Server_sankey.RData")

sankeyNetwork(
  Links = links, Nodes = nodes, Source = "source",
  Target = "target", Value = "value", NodeID = "name",
  units = "k€", fontSize = 10, fontFamily = "Arial Narrow", 
  nodeWidth = 30, LinkGroup = "col", NodeGroup = NULL, 
  margin = list(left = 220, right = 0), iterations = 256,
  colourScale = JS("d3.scaleOrdinal(d3.schemeCategory20c);")
)

Error in gsub("</", "\\u003c/", payload, fixed = TRUE) : 
  input string 1 is invalid UTF-8

---

On my 2nd question, about file encoding, either I missed something in the answer, or I wasn't clear enough in the question. I meant that if I have a file for which I have no idea what it's encoding is, how do I find it out (so that i.e. I can set it as a parameter in the source command)

SQL_Server_sankey_2.zip SQL_Server_sankey_1.zip SQL_Server_sankey_3.zip

[ghost]

On the Sybase ASE connection case

First and foremost, I want to emphasize that, using R-4.1.2, I do not need to perform any character conversion, at any point during the process.

• If I use R-4.2.2 and convert all the queries, stringi::stri_encode seems to work the same as iconv. But, when I tried to knit a really complicated rmd report, even if the result sets from the db seem to be correct, I get a strange error in a function call, where there has never been a problem before, and which I have no desire to try to "debug" it.

• If I omit the encoding = "CP1253" from dbConnect, all the results from all the queries should be processed, like this

df %>% 
    mutate(across(where(is.character), ~iconv(., from="CP1253", to="UTF-8")))

Zum Schluss: Ich bleibe in R-4.1.2 und bin zufrieden. Ich frage mich, ob ich der Einzige bin, der diese Probleme hat.

I've been trying to figure out for a while why you used the dative after "nach". Until I finally remembered that it is one of the 9 dative prepositions.... I need to put in a lot more effort.

---

Regarding communication, if you'd like, you can send me an email from an email account that I can access, if I need to get in touch with you. It would be greatly appreciated.

[fraupflaume]

In response to the first post you made:

I went back to look and see if it was something like an operator error, something that was dependent on the order I listed the utilities, or any other reason that it worked for you and not me. I found nothing different...but something is DEFINITELY different because it worked without an issue. Sigh. I'll chalk it up to operator error. I've updated the rpivotTable package again!

I'm not sure what you meant in your German statement...secrets? I'm confused! Tut mir leid! Erklär?

Völlig off-topic: Ich dachte nur, ich erwähne mal mein deutsches Lieblingswort. Ich finde es einfach lächerlich lang: Geschwindigkeitsbegrenzung. Es wird häufiger als Tempolimit bezeichnet.

I must have been pretty young, definitely under the age of 10, when I learned the endearment sweetie in German. It was more along the lines of someone said it and I asked what it meant. So some time later (same day? same week? months later?? I don't know), I wanted to use it when speaking to my sister. Only I said it wrong... So, the word that was said for sweetie was a derivation of Schatz (treasure), like Schatzie. What I said was Schwanzie... stick that in a translator (I'm not referring to a tail, either)...I know there was TON of laughing. Now that I'm much older I use it with my sister as an inside joke. At the time, I think they told me I was saying pig or piggy (that's Schwein...not Schwanz!). I was just a little girl. I'm glad they spared me the utter mortification had I really know what I said!

[ghost]

My statement was supposed to mean: I like mysteries but not that much. Because of all these strange things that happen with the software we use.

My favorite word is küssen. I think the use of umlaut here fits perfectly with the action the word describes.

I am only in my 4th month of taking German and I have a lot of questions. You don't want me to start asking you about them too. :)

Anyway, just once. Mal is a very common word but in some cases I don't know how to translate it. What does it mean in the sentence you used it in?

[fraupflaume]

I'm actually trying to work through all of the interesting things you've given me! (Encoding & all that...)

However, I just noticed this and wanted to respond to what you wrote most recently. (I'm still coming back to all the rest!) The word mal can go just about anywhere (like just in this sentence). You can always narrow down what it means a bit by whether its capitalized. It can be a noun or an adverb. All nouns are capitalized in German.

I'm sure you're aware of the literal meaning - once, or one time, times, something along those lines. However, it's used for emphasis in what I wrote. I just want to tell you my favorite word. or I just thought I'd just tell you what my favorite word is. Really though, because of Ich dachte nur I really just doesn't sound right with out mal. It is still grammatically correct if I write Ich dachte nur, ich erwähne mein deutsches Lieblingswort, though. (You don't need mal...well, ever.)

Thinking back, I think my Oma used the word doch in every other sentence. Doch is another word that doesn't really mean anything most of the time (just like really in this sentence).

[ghost]

What a fantastic teacher you are!

Ach und könnte ich doch nur ein einziges Mal die Uhren rückwärts drehen...

This song has long been a favorite of mine. I like it a lot more now that I can figure out the lyrics.

Here is another small gift for you. This is the website for “Advanced R Solutions” which provides solutions to the exercises from Hadley Wickham’s Advanced R, 2nd edition.

[fraupflaume]

I have found no method that will tell me unequivocally what the encoding is for a file. It seems like no matter what it wants to tell me whatever my native encoding is—so annoying!

Alright, I spent a while looking at the Sankey Rdata files. I'm sure I see something different than what you see because of the local set for R/Rstudio.

That being said, I'm not sure what's different about the 3rd zip file's 5th node. I can tell you that this is the 5th observation of each of the Rdata files' nodes (while my local is still "en"):

"Συστ. ΣΗΜ/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Θεσ (εκτος Λιαν-Δομ),\xce" "Συστ. ΣΗΜ/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Θεσ (εκτος Λιαν-Δομ),Θεσ-Πολυκ,Θεσ-Προμ" "Συστ. ΣΗΞ\u009c/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Ξ\u0098ΞµΟƒ (εκτος Λιαν-Δομ),Ξ"

What a hot mess! I thought this brought about an interesting way of determining the encoding of a file (although, obviously I have a good guess already). What happened is that it assumed the encoding is already UTF-8.

I can't take the third line and use either iconv or stri_encode or any other function because there is a mishmash of encoding in this single string. I have to break it down into bytes. I could use this extraction call to break strings down. That's assuming no data was lost when this was brought in. Are the three lines supposed to be identical?

str_extract_all(notRight, "Ξ.(?=[^\\\\])|Ξ.$|Ξ\\\\u....|Ο.")[[1]]

Since all through RData files had links and nodes, I numbered them like nodes0. I used this to rebuild that nodes that were shown like that third line:

library(tidyverse)
fixer <- function(notRight) {
  wrong <- str_extract_all(notRight, "Ξ.(?=[^\\\\])|Ξ.$|Ξ\\\\u....|Ο.")[[1]]
  fixed <- data.frame(original = wrong) %>% 
    mutate(correct = ifelse(!is.na(iconv(original, from = "UTF-8", to = "CP1253")),
                            iconv(original, from = "UTF-8", to = "CP1253"),
                            ifelse(!is.na(iconv(original, from = "UTF-8", to = "ISO-8859-7")),
                                   iconv(original, from = "UTF-8", to = "ISO-8859-7"),
                                   NA)))
  right <- reduce2(fixed$original, fixed$correct, .init = notRight, str_replace_all)
  print(right)
} 

nodes3 <- data.frame(name = invisible(lapply(1:length(nodes2$name), 
                                             function(j) fixer(nodes2$name[j]))) %>% 
                       unlist(use.names = F))

This is what I've got for line 5 now. I'm pretty sure something was dropped, because I'm guessing this line should end with that letter. I noticed that for a few other lines, as well (i.e., 'ΘεσσαλονίκΞ', which should have $\eta$ at the end instead, right? If so, then some data was lost. This encoding error should have "Ξ®" that would turn into $\eta$ when properly encoded.

"Συστ. ΣΗΜ/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Θεσ (εκτος Λιαν-Δομ),Ξ"

I did find that iconv was far more successful far more often than any other encoding function I used when I was messing around with this content.

I ended up building a frame of reference for this, but I didn't actually use it. You might be able to use it if things are particularly ugly (in terms of encoding discovery).

library(rvest)
html <- read_html("https://www.key-shortcut.com/en/writing-systems/abg-greek-alphabet")
thatsIt <- html %>% html_nodes("table") %>% html_table()

Gr <- thatsIt[[3]]
Grel <- thatsIt[[4]]
gr_el <- data.frame(Unicode = c(unlist(Gr, use.names = F),
                             unlist(Grel, use.names = F))) %>% 
  mutate(uni = substring(Unicode, 1, 6),
         letter = substring(Unicode, 7, 7),
         description = substring(Unicode, 8)) %>% 
  separate(description, sep = "HTML", into = c("description", "decimal", "hex")) %>% 
  mutate(decimal = str_extract(decimal, "[0-9]+"),
         hex = substring(stri_reverse(hex), 2, 6) %>% stri_reverse()) %>% 
  select(-Unicode)

filter(gr_el, description == "")
gr_el[gr_el$uni == "U+1F1C", "description"] <- "GREEK CAPITAL LETTER EPSILON WITH PSILI AND OXIA"
gr_el[gr_el$uni == "U+1F1D", "description"] <- "GREEK CAPITAL LETTER EPSILON WITH DASIA AND OXIA"

gr_el$ISOtoUTF8 <- iconv(gr_el$letter, from = "ISO-8859-7", to = "UTF-8")
gr_el$CPtoUTF8 <- iconv(gr_el$letter, from = "CP1253", to = "UTF-8")
gr_el <- gr_el %>% filter(!str_detect(description, "COPTIC"))

comment(gr_el) <- paste0("The column `ISOtoUTF8` is what the letter looks like if the ",
                         "encoding was converted from ISO-8859-7 to UTF-8. ",
                         "The column `CPtoUTF8` is what the letter looks like if the the ",
                         "encoding was converted from CP1253 to UTF-8.")

I also wrote a function that tests two strings, like what encoding took "Μ" and gave me "Ξ\u009c".

I went back to what I wrote that mentioned nach. I keep looking at it. It doesn't really translate correctly in English. You could say Das klingt wie eine tolle idee. For English, that translation is more understandable.

[ghost]

Thank you very much for the effort you have put in. I admit that (at least at the moment) I'm having a hard time following what you're doing and I don't know which question to answer first (If something really needs to be answered and the questions are not rhetorical).

What I suspect is that R (after the upgrade) misdiagnoses the encoding and either does a conversion (when it doesn't need to), or it doesn't (while it needs to). But, this appears to be too critical to not have been diagnosed (and corrected). I don't know what to assume.

This is the match that applies to me:

• SQL_Server_sankey_1.zip: R 4.1.2, (connection with encoding ON in dbConnect) Συστ. ΣΗΜ/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Θεσ (εκτος Λιαν-Δομ),Θεσ-Πολυκ,Θεσ-Προμ

• SQL_Server_sankey_2.zip: R 4.2.2, (connection with encoding ON in dbConnect): Συστ. ΣΗΞ\u009c/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Ξ\u0098ΞµΟƒ (εκτος Λιαν-Δομ),Ξ

• SQL_Server_sankey_3.zip: R 4.2.2, (connection with encoding OFF in dbConnect): Συστ. ΣΗΜ/ΣΗΣ-ΤΗΛ/ΣΗΣ & ETCS (Εγκ.,Λειτ.,Συντ.): Αθη-Θεσ (εκτος Λιαν-Δομ),\xce

The first one is the correct string. In the other two cases the last part of the string, i.e. "Θεσ-Πολυκ,Θεσ-Προμ", is lost. And that "\xce", in the 3rd case, causes the program to crash.

P.S. I have just read about this new book. The article said that: "Every computer science teacher should get a copy to show to their students". Since it's okay if I don't understand it, I might try reading it, I suppose.

[ghost]

I think this S.O. question refers to the same problem. And this one too. Here's another one that also seems relevant.

Another github issue on R-4.2 encoding problems.

[fraupflaume]

Have you gone this route yet? There's someone specifically for handling R and translation issues for Greek. https://developer.r-project.org/TranslationTeams.html

[ghost]

Hmm, I think I know him. I see we're also connected on linkedin. I'll give it a shot.

I've tried to contact Panagiotis-kanavos (He appears in the 2nd link in my previous message, he is also Greek and seems to be experienced enough). I sent him a message and explained my problem (that happened before contacting you), but unfortunately he never replied. (You see, not everyone is like you...)

[fraupflaume]

I had a whole big message typed out, I thought I was really onto something...but I really wasn't.

However, since you keep providing interesting resources, have you seen or heard of this tool? https://github.com/ksint/pasteLastVal It's almost better than sliced bread. I was just thinking about how much I missed it while I was using RStudio in my VirtualBox.

[ghost]

(I just saw your email, but I haven't had a chance to read it, because I was writing this. Take a look at that. ) Do you remember this comment? Check out what I found out today !!!!!!!

It's about the SQL SERVER connection

If instead of odbc::odbc() use RODBCDBI::ODBC() THERE IS NO PROBLEM. I suppose you could check it yourself, using this query

library(DBI)
# con <- dbConnect(odbc::odbc(), encoding = "CP1253", "ODYSSEUS_PM_64", Uid="sa", Pwd="sqlSQL")
con <- dbConnect(RODBCDBI::ODBC(), dsn="ODYSSEUS_PM_64", user="sa", password="sqlSQL")
# dbDisconnect(con)  

# con <- dbConnect(odbc::odbc(), encoding = "CP1253",  .connection_string = "Driver={ODBC Driver 17 for SQL Server};server=DESKTOP-61HASAC;database=ergose2;Uid=sa;Pwd=sqlSQL;")

dbGetQuery(con, paste("                  
              SELECT  *
              FROM    [ergose2].[dbo].[ACTVCODE]
              WHERE   ([parent_actv_code_id] = 10128 AND [actv_code_id] = 10126)",sep=""))

[ghost]

I have also tried RODBC. It works too, and provides a parameter to declare db encoding (Moreover, it is continuously and actively updated by its development team). So I tried it with the ASE db. The only problem is in correctly sending queries that include Greek. This seems to be fixed if I replace the dbGetQuery function with a UDF like this. I still need to test if all my reports work ok.

My_Get_Query <- function(con, sql_statement) {
   RODBC::sqlQuery(con, iconv(sql_statement, "CP1253", "UTF-8")) #works
 }