fix original encoding in getSymbolList()

dkibalnikov commented 1 year ago

Issue

getSymbolList("Finam") downloads file with original "WINDOWS-1251" encoding. The function readLines(encoding = 'UTF-8') does not help since fun doc contains: "it is not used to re-encode the input. ". So, after R update up to:

R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)

getSymbolList() started to throw error:

Error in sub("var .*?= \\[", "", fr[2]) : input string 1 is invalid
In addition: Warning message:
In sub("var .*?= \\[", "", fr[2]) :
  unable to translate 'var aEmitentNames = ['SPFB.Eu-12.22','SPFB.BR-9.22','SPFB.Eu-3.23','SPFB.BR-10.22','SPFB.BR-11.22','US1.LOGM_disappeared_2835723','SPFB.MXI-12.22','SPFB.RUON-12.22','US1.TPG','SPFB.BR-3.22-4.22','SPFB.FNI-9.22','SPFB.AFLT-12.22','SPFB.GAZR-3.23','SPFB....' to a wide string

Fix

Error could be easily fixed by forcing convert from "WINDOWS-1251" to "UTF8" via iconv() function.

Caveats

I've tested on MacOS(ARM) and MacOS(Intel) and everything looks fine. I assume probability to face issue for Windows. Therefore it's better to test on Windows before fork merge .
Other sources also could use unwanted "WINDOWS-1251", so other fixes might be considered the same way

mytarmail commented 1 year ago

i have same problem on windows

getSymbols('Сбербанк',src='Mfd')

No encoding supplied: defaulting to UTF-8.
No encoding supplied: defaulting to UTF-8.
Error in `[.data.frame`(fr, , (5:10)) : undefined columns selected
In addition: There were 27 warnings (use warnings() to see them)

getSymbols('LKOH',src='Finam') 
trying URL 'https://www.finam.ru/cache/N72Hgd54/icharts/icharts.js'
downloaded 1.2 MB

Error in readLines(con = tmp, warn = FALSE, encoding = "UTF-8") : 

  error reading from the connection
In addition: Warning message:
In readLines(con = tmp, warn = FALSE, encoding = "UTF-8") :
  invalid or incomplete compressed data

DrDinhLuong commented 1 year ago

I install rusquant

So, when I run this code, the problems showed that:

alpha_performance = getSymbolList(src='Rusquant', api.key=rusquant_token) Error in getSymbolList(src = "Rusquant", api.key = rusquant_token) : could not find function "getSymbolList"

Pls help me.