lionel-
commented
3 years ago What version of ESS are you running? Can you post a minimal reproducible example please?
Closed gpetrini closed 3 years ago
lionel-
commented
3 years ago What version of ESS are you running? Can you post a minimal reproducible example please?
lionel-
commented
3 years ago Are you sure it "crashes"? What do you mean by that?
gpetrini
commented
3 years ago ESS version: 18.10.3
The file in which I get the error is this. To reproduce the behavior just need to unzip the PNAD_COVID_112020.zip and evaluate the .org file.
After run R src-block I got the following message:
Code block evaluation complete
And then it freezes emacs and I cannot interrupt or anything else, so I need to run sudo xkill to stop it. I could be related to the dataset size (115 mb)?
Next, I am pasting my R configs
(after! ess
(set-popup-rule! "^\\*R" :ignore t)
;; HACK the below is not working as expected I think becase of the way doom
;; manages windows and frames
;; (set-popup-rules!
;; ;; '(("^\\*R Dired" :slot 1 :vslot -1 :size #'+popup-shrink-to-fit)
;; ;; ("^\\*R" :side 'right :slot 1 :vslot -1 :select f)
;; ;; ("^\\*Help" :slot -1 :vslot -2 :select t :ttl 0))
;; `(("*R Dired"
;; (display-buffer-reuse-window display-buffer-in-side-window)
;; (side . right)
;; (slot . -1)
;; (window-width . 0.33)
;; (reusable-frames . nil))
;; ("*R"
;; (display-buffer-reuse-window display-buffer-in-side-window)
;; (side . right)
;; (window-width . 0.5)
;; (reusable-frames . nil))
;; ("*Help"
;; (display-buffer-reuse-window display-buffer-below-selected)
;; (side . left)
;; (slot . 1)
;; (window-width . 0.33)
;; (reusable-frames . nil)))
;; ;; '(
;; ;; ("^\\*Compil\\(?:ation\\|e-Log\\)" :size 0.3 :ttl 0 :quit t))
;; )
)
(use-package! ess
;; :ensure t
:demand t
:init
(require 'ess-site)
:config
(setq display-buffer-alist
`(("*R Dired"
(display-buffer-reuse-window display-buffer-in-side-window)
(side . right)
(slot . -1)
(window-width . 0.33)
(reusable-frames . nil))
("*R"
(display-buffer-reuse-window display-buffer-in-side-window)
(side . right)
(window-width . 0.5)
(reusable-frames . nil))
("*Help"
(display-buffer-reuse-window display-buffer-below-selected)
(side . left)
(slot . 1)
(window-width . 0.33)
(reusable-frames . nil)))
)
(setq ess-style 'RStudio
;; auto-width
ess-auto-width 'window
;; let lsp manage lintr
ess-use-flymake nil
;; Stop R repl eval from blocking emacs.
ess-eval-visibly 'nowait
ess-use-eldoc nil
ess-use-company nil
)
(setq ess-use-flymake nil)
(setq ess-r--no-company-meta t)
(setq ess-ask-for-ess-directory t
ess-local-process-name "R"
ansi-color-for-comint-mode 'filter
comint-scroll-to-bottom-on-input t
comint-scroll-to-bottom-on-output t
comint-move-point-for-output t)
;; insert pipes etc...
(defun tide-insert-assign ()
"Insert an assignment <-"
(interactive)
(insert " <- "))
(defun tide-insert-pipe ()
"Insert a %>% and newline"
(interactive)
(insert " %>%"))
;; set keybindings
;; insert pipe
(define-key ess-r-mode-map (kbd "M-s-'") 'tide-insert-assign)
(define-key inferior-ess-r-mode-map (kbd "M-s-'") 'tide-insert-assign)
;; insert assign
(define-key ess-r-mode-map (kbd "M-s-\"") 'tide-insert-pipe)
(define-key inferior-ess-r-mode-map (kbd "M-s-\"") 'tide-insert-pipe)
)
;; ess-view
;; open a df in an external app
;; TODO need to find out how to display options vertically
(use-package! ess-view
;; :ensure t
:after ess
:diminish
:config
(setq ess-view--spreadsheet-program "open") ; open in system default on macos
(setq ess-view-inspect-and-save-df t)
;; enable ess-view package to be triggered from the source doc
;; see: <https://github.com/GioBo/ess-view/issues/9>
(defun ess-view-extract-R-process ()
"Return the name of R running in current buffer."
(let*
((proc (ess-get-process)) ; Modified from (proc (get-buffer-process (current-buffer)))
(string-proc (prin1-to-string proc))
(selected-proc (s-match "^#<process \\(R:?[0-9]*\\)>$" string-proc)))
(nth 1 (-flatten selected-proc))
)
)
:bind
(
("C-c C-e C-v" . ess-view-inspect-df)
;; the below doesn't work on osx
;; see <https://github.com/GioBo/ess-view/issues/5>
;; ("C-x q" . ess-view-inspect-and-save-df)
)
)
(map! :leader
:prefix "m"
"cv" #'ess-view-inspect-df
"cc" 'ess-tide-insert-chunk
"w" 'ess-eval-word
)
(use-package! ess-view-data
:load-path "./ess-view-data"
:after ess
:init
(require 'ess-view-data))
;; ===========================================================
;; Polymode
;; ===========================================================
;; basic polymode
(use-package! polymode
;; :ensure t
:config
(use-package! poly-R)
(use-package! poly-markdown)
(use-package! poly-noweb)
(add-to-list 'auto-mode-alist '("\\.md" . poly-markdown-mode))
(add-to-list 'auto-mode-alist '("\\.Snw" . poly-noweb+r-mode))
(add-to-list 'auto-mode-alist '("\\.Rnw" . poly-noweb+r-mode))
(add-to-list 'auto-mode-alist '("\\.Rmd" . poly-markdown+r-mode))
;; Export files with the same name as the main file
(setq polymode-exporter-output-file-format "%s")
)
;; for use in Rmarkdown documents
(use-package! markdown-mode
:config
(define-key markdown-mode-map (kbd "M-s-'") 'tide-insert-assign)
(define-key markdown-mode-map (kbd "M-s-\"") 'tide-insert-pipe)
)
;; ===========================================================
;; IDE Functions
;; ===========================================================
;; Bring up empty R script and R console for quick calculations
(defun ess-tide-scratch ()
(interactive)
(progn
(delete-other-windows)
(setq new-buf (get-buffer-create "scratch.R"))
(switch-to-buffer new-buf)
(R-mode)
(setq w1 (selected-window))
(setq w1name (buffer-name))
(setq w2 (split-window w1 nil t))
(if (not (member "*R*" (mapcar (function buffer-name) (buffer-list))))
(R))
(set-window-buffer w2 "*R*")
(set-window-buffer w1 w1name)))
(global-set-key (kbd "C-x 9") 'ess-tide-R-scratch)
;; Not sure if need this as plymode has something similar
(defun ess-tide-shiny-run-app (&optional arg)
"Interface for `shiny::runApp()'. With prefix ARG ask for extra args."
(interactive)
(inferior-ess-r-force)
(ess-eval-linewise
"shiny::runApp(\".\")\n" "Running app" arg
'("" (read-string "Arguments: " "recompile = TRUE"))))
;; Graphics device management ;;
(defun ess-tide-new-gdev ()
"create a new graphics device"
(interactive)
(ess-eval-linewise "dev.new()"))
(defun ess-tide-cur-gdev ()
"return current graphics device"
(interactive)
(ess-eval-linewise "dev.cur()"))
(defun ess-tide-list-all-gdev ()
"list all graphics devices"
(interactive)
(ess-eval-linewise "dev.list()"))
(defun ess-tide-switch-to-gdev ()
"Prompt for the number of the graphics device to make current"
(interactive)
(setq dev-num
(read-from-minibuffer "Select R graphics device: "
nil nil t t "1"))
(ess-eval-linewise
(format "dev.set(%s)" dev-num)))
(defun ess-tide-switch-next-gdev ()
"switch to next available graphics device"
(interactive)
(ess-eval-linewise "dev.set(dev.next())"))
(defun ess-tide-switch-prev-gdev ()
"switch to previous available graphics device"
(interactive)
(ess-eval-linewise "dev.set(dev.prev())"))
(defun ess-tide-save-gdev-pdf ()
"Save current graphics device as pdf"
(interactive)
(ess-eval-linewise "dev.copy2pdf()"))
(defun ess-tide-capture-gdev ()
"Capture current graphics device as image"
(interactive)
(ess-eval-linewise "dev.capture()"))
;; Devtools
(defun ess-tide-devtools-setup ()
"setup R package in current working directory"
(interactive)
(ess-eval-linewise "devtools::setup()"))
;;======================================================================
;; (R) markdown mode
;;======================================================================
;; Insert new chunk for Rmarkdown
(defun ess-tide-insert-chunk (header)
"Insert an r-chunk in markdown mode."
(interactive "sLabel: ")
(insert (concat "```{r " header "}\n\n```"))
(forward-line -1))
(global-set-key (kbd "C-c M-i") 'ess-tide-insert-chunk)
;; Mark a word at a point ==============================================
;; http://www.emacswiki.org/emacs/ess-edit.el
(defun ess-edit-word-at-point ()
(save-excursion
(buffer-substring
(+ (point) (skip-chars-backward "a-zA-Z0-9._"))
(+ (point) (skip-chars-forward "a-zA-Z0-9._")))))
;; eval any word where the cursor is (objects, functions, etc)
(defun ess-eval-word ()
(interactive)
(let ((x (ess-edit-word-at-point)))
(ess-eval-linewise (concat x)))
)
;; key binding
(define-key ess-mode-map (kbd "C-c r") 'ess-eval-word)
;; (load! "+ess.el")Additionally, I am pasting a minimal file which I got the same behavior.
#+TITLE: Ess issue 1092
#+PROPERTY: header-args python :results output drawer :eval never-export :session Cecon
#+PROPERTY: header-args R :results output drawer :eval never-export :session Cecon
* Python
** Loading packages
#+BEGIN_SRC python
import pandas as pd
import numpy as np
from datetime import datetime as dt
from datetime import timedelta
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
from matplotlib.ticker import FuncFormatter
import seaborn as sns
start_year = "2018-01-01"
base = "2014-12-01"
def line_format(label):
"""
Convert time label to the format of pandas line plot
"""
month = label.month_name()[:3]
if month == "Jan":
month += f"\n{label.year}"
return month
# ax.set_xticklabels(map(lambda x: line_format(x), df[start_year:].index))
def line_format_day(label):
"""
Convert time label to the format of pandas line plot
"""
month = label.month_name()[:3]
day = label.day
string = f"{day}/{month}"
return month
def interpolator(df):
for col in df:
df[col] = pd.to_numeric(df[col], errors="coerce")
df = df.resample("D").interpolate(method="time")
return df
def consulta_bc(codigo_bcb, nome=["Nome da série"]):
url = "http://api.bcb.gov.br/dados/serie/bcdata.sgs.{}/dados?formato=json".format(
codigo_bcb
)
df = pd.read_json(url)
df["data"] = pd.to_datetime(df["data"], dayfirst=True)
df.set_index("data", inplace=True)
df.index.name = ""
df.columns = nome
return df
#+END_SRC
** Example python code
#+BEGIN_SRC python :results output org table
df = consulta_bc(24364, ["IBCBr (%YoY)"])
df.index.name = ""
# df = rebase(df)
df = df.pct_change(12)
df = (1 + df / 100).rolling(window=12).agg(lambda x: x.prod()) - 1
df = df[start_year:]
df = interpolator(df)
print(df.head())
#+END_SRC
#+RESULTS:
#+begin_src org
IBCBr (%YoY)
2018-01-01 0.001847
2018-01-02 0.001849
2018-01-03 0.001851
2018-01-04 0.001853
2018-01-05 0.001855
#+end_src
* R
** Dowloading dataset (115 MB)
#+begin_src shell
wget ftp://ftp.ibge.gov.br/../Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_PNAD_COVID19/Microdados/Dados/PNAD_COVID_112020.zip
unzip PNAD_COVID_112020.zip
#+end_src
** Loading packages
#+begin_src R
library(dplyr)
library(srvyr)
library(readr)
library(ggplot2)
library(Cairo)
library(cowplot)
library(magick)
#+end_src
** Loading the dataset
#+begin_src R
pnad_covid <- read_csv("./PNAD_COVID_112020.csv", col_types = cols(.default = "d"))
mes <- "Novembro"
# ligando Pesos e filtrando Salvador
pnad_covid <- pnad_covid %>%
as_survey_design(
ids = UPA,
strata = Estrato,
weights = V1032,
nest = TRUE
)
# Criando colunas com Variáveis
pnad_covid <- pnad_covid %>% mutate(one = 1,
Sexo = ifelse(A003 == 1, "Homem", "Mulher"),
Idade = case_when(
A002 %in% 15:24 ~ "15-24",
A002 %in% 25:34 ~ "25-34",
A002 %in% 35:49 ~ "35-49",
A002 %in% 50:64 ~ "50-64",
A002 > 64 ~ "65+"),
Cor = case_when(
A004 == 1 ~ "Branca",
A004 == 2 ~ "Preta",
A004 == 4 ~ "Parda"),
Escolaridade = factor(case_when(
A005 %in% 1:2 ~ "Sem Instrução ou Fundamental Incompleto",
A005 %in% 3:4 ~ "Fundamental completo ou Médio Incompleto",
A005 %in% 5:6 ~ "Médio completo ou Superior Incompleto",
A005 == 7 ~ "Superior completo",
A005 == 8 ~ "Pós-graduação"),
levels = c( "Sem Instrução ou Fundamental Incompleto",
"Fundamental completo ou Médio Incompleto",
"Médio completo ou Superior Incompleto",
"Superior completo",
"Pós-graduação")),
Tipo_emprego = factor(case_when(
C007 == 1 ~ "Trabalhador doméstico (empregado doméstico, cuidados, babá)",
C007 == 2 ~ "Militar",
C007 == 3 ~ "Policial ou Bombeiro",
C007 == 4 ~ "Setor privado",
C007 == 5 ~ "Setor público",
C007 == 6 ~ "Empregador",
C007 == 7 ~ "Autônomo (Conta própria)"),
levels = c( "Trabalhador doméstico (empregado doméstico, cuidados, babá)",
"Militar",
"Policial ou Bombeiro",
"Setor privado",
"Setor público",
"Empregador",
"Autônomo (Conta própria)")),
Faixa_salario = factor(case_when(
C01012 <= 1044 ~ "Menos de um salário mínimo",
C01012 %in% c(1045:2090) ~ "Entre 1 e 2",
C01012 %in% c(2091:3135) ~ "Entre 2 e 3",
C01012 %in% c(3136:4180) ~ "Entre 3 e 4",
C01012 %in% c(4181:5225) ~ "Entre 4 e 5",
C01012 >= 5226 ~ "Mais de 5"),
levels = c("Menos de um salário mínimo",
"Entre 1 e 2",
"Entre 2 e 3",
"Entre 3 e 4",
"Entre 4 e 5",
"Mais de 5")),
domicilio_situacao = factor(case_when(
F001 == 1 ~ "Próprio - já pago",
F001 == 2 ~ "Próprio - ainda pagando" ,
F001 == 3 ~ "Alugado",
F001 %in% 4:6 ~ "Cedido (Por empregador, Familiar ou outro)"),
levels = c("Próprio - já pago",
"Próprio - ainda pagando",
"Alugado",
"Cedido (Por empregador, Familiar ou outro)")),
home_office = ifelse(C013 == 1, "Home Office", "Presencial"),
auxilio_emergencial = ifelse(D0051 == 1, "Auxílio", "Sem auxílio")
)
############### Home office - Por sexo e cor ################### Criando dataset para conferir pessoas em Home Office
df <- pnad_covid %>%
group_by(Sexo, Cor) %>%
summarise(
home_office = survey_total(C013 == 1, na.rm = TRUE),
mao_de_obra = survey_total(C001 == 1, na.rm = TRUE)) %>%
mutate(trab_home_office = (home_office/mao_de_obra)*100) %>%
drop_na()# gráfico
fig <- ggplot(df, aes(fill = Cor, y = trab_home_office, x = Sexo)) +
geom_bar(position = "dodge", stat = "identity") +
geom_text(aes(label=sprintf("%1.2f%%",trab_home_office)),size = 3, position =position_dodge(width=0.9),
vjust=-0.5, color = 'black',fontface='bold') +
theme_classic() +
theme(axis.title.x = element_text(colour = "black"),
axis.title.y = element_text(colour = "black"),
axis.text.y = element_text(face="bold", color="#000000",
size=10),
axis.line = element_line(colour = "black",
size = 1, linetype = "solid"),
axis.text=element_text(size=6, face="bold"),
axis.text.x = element_text(face="bold", color="#000000", size=10),
plot.title = element_text(colour = "black", size = 17, hjust=0.5),
legend.position = "bottom", legend.background = element_rect(fill="ghostwhite", size=0.7, linetype="blank")) +
labs(x = "Sexo", fill = "Cor/Raça: ", caption = "Fonte: Microdados da Pnad Covid19 - IBGE. Novembro 2020.",
title = "Pessoas em home office, por cor/raça e sexo") +
scale_fill_manual(values = c("#00b894","#ff7675","#0984e3","#6c5ce7")) +
scale_y_discrete(limits=factor(0:100), breaks = c(0,10,20,30,40,50,60,70,80,90,100), name = "Percentual (%)")
#+end_src@gpetrini Please make your example minimal. It should be a few lines of org code and should ideally reproduce in a vanilla Emacs session (with command line argument -q so your config is not loaded).
gpetrini
commented
3 years ago Sorry about that. I tried to shrink my example in order to include a little python chunk and a R src-block which loads my dataset.
#+TITLE: Ess issue 1092
* Python
#+BEGIN_SRC python
import pandas as pd
def consulta_bc(codigo_bcb, nome=["Nome da série"]):
url = "http://api.bcb.gov.br/dados/serie/bcdata.sgs.{}/dados?formato=json".format(
codigo_bcb
)
df = pd.read_json(url)
df["data"] = pd.to_datetime(df["data"], dayfirst=True)
df.set_index("data", inplace=True)
df.index.name = ""
df.columns = nome
return df
df = consulta_bc(24364, ["IBCBr (%YoY)"])
print(df.head())
#+END_SRC
* R
** Dowloading dataset (115 MB)
#+begin_src shell
wget ftp://ftp.ibge.gov.br/../Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_PNAD_COVID19/Microdados/Dados/PNAD_COVID_112020.zip
unzip PNAD_COVID_112020.zip
#+end_src
** Loading packages
#+begin_src R
library(dplyr)
library(srvyr)
library(readr)
pnad_covid <- read_csv("./PNAD_COVID_112020.csv", col_types = cols(.default = "d"))
pnad_covid <- pnad_covid %>%
as_survey_design(
ids = UPA,
strata = Estrato,
weights = V1032,
nest = TRUE
)
# Criando colunas com Variáveis
pnad_covid <- pnad_covid %>% mutate(one = 1,
Sexo = ifelse(A003 == 1, "Homem", "Mulher"),
Cor = case_when(
A004 == 1 ~ "Branca",
A004 == 2 ~ "Preta",
A004 == 4 ~ "Parda")
)
############### Home office - Por sexo e cor ################### Criando dataset para conferir pessoas em Home Office
df <- pnad_covid %>%
group_by(Sexo, Cor) %>%
summarise(
home_office = survey_total(C013 == 1, na.rm = TRUE),
mao_de_obra = survey_total(C001 == 1, na.rm = TRUE)) %>%
mutate(trab_home_office = (home_office/mao_de_obra)*100) %>%
drop_na()# gráfico
#+end_srcI tried to run the above file in a vanila emacs session but I got the following message:
org-babel-execute-src-block: No org-babel-execute function for pythonThank you for trying to make your reprex minimal. However it's still not minimal enough and it does all sorts of things like downloading huge files. Please reduce your example. If I can tell what you were working on from the reprex it means it's not minimal enough.
Also can you describe the process for causing the hang? I don't use org-mode to evaluate chunks and I don't know what I should be doing. When I try C-c C-c on the chunks I don't see a hang.
Can you please try this (this is what a minimal example looks like) and see if it hangs for you please:
#+BEGIN_SRC python
1 + 1
#+END_SRC
#+begin_src R
1 + 1
#+end_srcSorry about this huge example. Everything works fine when I run you example. That's why I think this is related to my dataset size. Additionally, I can run my whole R script in RStudio (with this same dataset) but it stops at the very beginning when I try in emacs. This is why I think it is emacs or ess related and not a hardware problem.
lionel-
commented
3 years ago It may also be a problem in org-mode or in the ESS layer of org-mode. Note that the ESS layer of org-mode is implemented by different people than the ESS maintainers and we have no relationship with them.
If you think the data size has a meaningful role in your problem, please create a minimal example with artificially generated data (like data.frame(x = rep(NA, 1e6))).
You haven't replied to this question:
Also can you describe the process for causing the hang? I don't use org-mode to evaluate chunks and I don't know what I should be doing. When I try C-c C-c on the chunks I don't see a hang.
gpetrini
commented
3 years ago Sorry for the late response.
Here is a minimal example with your suggestions as follows:
#+TITLE: Ess issue 1092
#+PROPERTY: header-args python :results output drawer :eval never-export :session Cecon
#+PROPERTY: header-args R :results output drawer :eval never-export :session Cecon
* Python
#+BEGIN_SRC python
print(1+1)
#+END_SRC
#+RESULTS:
:results:
2
:end:
* R
#+begin_src R
df = data.frame(x = rep(NA, 1e6))
summary(df)
#+end_srcEverything worked as expected. I am not sure if it is related with dataset size and the operations employed (survey microdata).
Also can you describe the process for causing the hang? I don't use org-mode to evaluate chunks and I don't know what I should be doing. When I try C-c C-c on the chunks I don't see a hang.
The hang occurs after I run C-c C-c in org-mode. I've tried to run the buffer after hitting C-c ' and got the same freezing.
Since it works fine in RStudio I guess it is not a problem with my code.
Additionally, I have a ess module in my Doom's configuration. This is the best way to install ess? Should I include (use-package! ess) as well?
gpetrini
commented
3 years ago I am not really sure if was it that solved my issue, but here is what I've done.
My emacs config is based on doom. In my init.el file, I enabled ess module. However, I also had a (use-package! ess) chunk in my config.elin which I've defined some ESS configurations. After removing the (use-package! ess) part, I was able to run my script and it took a similar time to run as RStudio.
Since it seems to be a doom-related problem, I'll close the issue.
Thank you for the support and sorry for this mess
I am running a doom emacs based configuration as in this link. It works fine when I run a R chunk in org-mode. However, when I run the same code in an org-file with other python src-blocks it hangs forever and crashes.
Here is more detailed infos about my personal configs
Thank you in advance