search

rstudio / blogdown

Create Blogs and Websites with R Markdown
https://pkgs.rstudio.com/blogdown/
1.74k stars 330 forks source link

blogdown - github - netlify works for md, but not for rmd #169

Closed rainer-rq-koelle closed 7 years ago

rainer-rq-koelle commented 7 years ago

First of all: thanks and congrats, blogdown is a fantastic beast that made a guy like me thinking about hosting a website and move slowly into the 21st century.

I worked myself through the blogdown book/documentation with glimpses to some online tutorials, I managed to deploy a first stab (using the bootstrap v4 theme) via netlify (http://rqkoelle.netlify.com).

Adding new pages as md (e.g. about-me) and more blog posts as md files do allow to run the hugo build on netlify. However, if i use Rmd to make use of r in a blog this pipeline fails. In fact, the build is successfully completed, but the Rmd files are not converted into payload. While the files sit nicely in the post/ folder: https://github.com/rainer-rq-koelle/rq-netlify-blog/tree/master/content/post nothing ends up in the public folder that serves the site.

I understand that the build command obviously only uses hugo on netlify and not the magic of blogdown I can use locally.

Is there a way to achieve this via the netlify deploy as well? Or can i do something like blogdown::build_post() to just create the desired rmd to html conversion which can then be pushed to github?

My understanding from the first push of the local build_site() is that netlify seems to tolerate the html files created from the rmd in the public folder. I would just like to avoid having to push loads of files when doing a local build to ensure the rmd to html conversion on a single (or little number of) files.

Thanks.

tcgriffith commented 7 years ago

blogdown uses pandoc to convert rmd to html, i think you should commit those .html files knitted from rmd files to your repo, instead of pure rmd source file.

This is referred in the blogdown docs 1.5 rmarkdown vs markdown:

"You may have some intermediate HTML files in the source directory of your website, because blogdown has to call rmarkdown to pre-render .Rmd files .html"

And if you use rstudio, you can try addins- serve site to see if you site is displayed properly.

yihui commented 7 years ago

Yes, as @tcgriffith said, commit everything you see under content/ and static/, including the .html output files. More info on GIT.

liyi-1989 commented 7 years ago

I met a similar problem. I tried the following two ways:

and push the results in the Github repo, and let Netlify to deploy it. What I get:

Did I do something wrong? Thank you very much!

Update:

If I manually rename .markdown to .md, then the post will appear. However, the R plot figure cannot be displayed (manually change the location of the folder figure?).

I use HUGO 0.30.2 (in order to compile the academic theme).

tcgriffith commented 7 years ago

rmarkdown to markdown is used when you do not have r chunks in your post. so if you have those, you should stick to html

you should also preview the site locally before pushing it to deploy. in rstudio, select addins-serve site to see if the site looks correct. commit and push everything to github, except the public folder netlify should render the site exact as you previewed locally

liyi-1989 commented 7 years ago

Thanks for your answer! Unfortunately, I am a windows user. I saw from the blowdown manual, the preview function is beta in windows. I am one of this. When I preview it, R will die after 2-3 seconds. (I don't know why) So I have to make a lot of commits and deployment to preview it on netlify. Good to know that it should not have R chunks. I will try rmd again tomorrow.

byzheng commented 7 years ago

Do you preview in rstudio? If so, preview in a separate R GUI.

On 5 Nov 2017 15:18, "Yi Li" notifications@github.com wrote:

Thanks for your answer! Unfortunately, I am a windows user. I saw from the blowdown manual, the preview function is beta in windows. I am one of this. When I preview it, R will die after 2-3 seconds. (I don't know why) So I have to make a lot of commits and deployment to preview it on netlify. Good to know that it should not have R chunks. I will try rmd again tomorrow.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rstudio/blogdown/issues/169#issuecomment-341949938, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlKJRHROGwJmBsErrEUqyrMkqKRZtzIks5szUU5gaJpZM4OlHR9 .

tcgriffith commented 7 years ago

I have no problem previewing my site in windows10. maybe you should fire up a new issue and provide more info about your problem.

you can use cmd or powershell to run hugo

liyi-1989 commented 7 years ago

Thanks @byzheng ! I will try that.

Thanks TC! Good to know it works on your windows 10. I will double check this tomorrow to make sure I configured in the right way.

yihui commented 7 years ago

@liyi-1989 Could you try this? https://bookdown.org/yihui/blogdown/livereload.html

install.packages(c("processx", "later"))
options(blogdown.generator.server = TRUE)
liyi-1989 commented 7 years ago

@yihui This helps. It works now. Thanks!

andkov commented 7 years ago

Thanks, @yihui. I'm confirming that 

install.packages(c("processx", "later"))
options(blogdown.generator.server = TRUE)

resolves the issue of RStudio crashing. Even when I'm serving the site from the same .Rproj ( via Serve Site addin)

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12    bookdown_0.5    later_0.6       digest_0.6.12   rprojroot_1.2   mime_0.5       
 [7] backports_1.1.1 magrittr_1.5    evaluate_0.10.1 blogdown_0.2.2  rlang_0.1.2     stringi_1.1.5  
[13] rmarkdown_1.7   tools_3.4.1     servr_0.8       stringr_1.2.0   httpuv_1.3.5    yaml_2.1.14    
[19] compiler_3.4.1  htmltools_0.3.6 knitr_1.17
andkov commented 7 years ago

@yihui 

> (blogdown:::Rscript(c("-e", shQuote("httpuv::startDaemonizedServer('127.0.0.1', p <- servr:::random_port(), list(call = function(req) list(status = 200L, body = 'Hello world!', headers = list('Content-Type' = 'text/plain')))); readLines(paste0('http://127.0.0.1:', p), warn = F)"))))
[1] "300383456"
[1] "Hello world!"
[1] 0
DataStrategist commented 6 years ago

fwiw, I couldn't get my Rmd to show up online even tho it was rendering locally... but when I opened up the file, it seems it had added an implicit draft:yes clause image

I resolved by hardcoding draft:no in the Rmd header... that got the file online

yihui commented 6 years ago

@DataStrategist See #312.

yihui commented 6 years ago

@DataStrategist Did you see my reply above https://github.com/rstudio/blogdown/issues/169#issuecomment-321460204?

DataStrategist commented 6 years ago

@yihui I've looked at both your comments here. In terms of am I committing EVERYTHING, the answer is yes... except where the .gitignore prohibits (like public). Anyway, I have pushed everything and not changed any of the .gitignore info.

I have also tried to the options under Live-Reload... but to no avail. I've also tried to blogdown::hugo_build() instead of blogdown::build_site() but to no avail. The repo is: https://github.com/DataStrategist/amitkohli.com and the live site is: http://amitkohli.com/ The blog post that is the Rmd is the first one... 'Crime statistics in london'. I just committed the one w/ the broken format. Hope you like my commit message :) And thanks for your help!

yihui commented 6 years ago

@DataStrategist Weird. There should be a rmarkdown-libs folder under https://github.com/DataStrategist/amitkohli.com/tree/master/static. The missing folder is the reason why your leaflet map cannot be rendered.

yihui commented 6 years ago

Another thing is, I recommend you to try "Serve site" (https://bookdown.org/yihui/blogdown/workflow.html), and make sure things work at least locally before pushing to Github. Pushing blindly to Github + waiting for Netlify to build the site may not be fun (and I won't be surprised if you Grrrrr :).

DataStrategist commented 6 years ago

Yeah, it's all working now! So the culprit was the .gitignore in /static... as it was IGNORE ALLing which means that indeed none of the packages were being pushed to netlify. You can see here what the "out of the box" configuration is: https://github.com/DataStrategist/amitkohli.com/blame/06a60c1149b3ecc8abdd1c3286d4dfcf2717c038/static/.gitignore . Maybe that .gitignore shouldn't exist at all in fact?

I do check the serve site... it was working locally because the libraries were all there :) It just wasn't pushing online...

Anyway I will stop GRRRR ing now and will start singing your praises and actually finish that blog post!

yihui commented 6 years ago

Oh, actually I had looked at your .gitignore (more than once), but the asterisk * was so small and looked like #, so I overlooked it. I thought the three lines were all comments, and my eyes automatically ignored comments when scanning this file! This .gitignore does not sound necessary to me.

Very glad that your problem was solved!

rikudoukarthik commented 2 years ago

I am also having an issue with .Rmd posts. I have gone through several forum posts including issues here, but have been unable to reach a working solution, perhaps because I am new to all this.

For my personal website (repo here), I am using the Hugo Apero theme by Alison Hill, and am struggling with rendering .Rmd blog posts. I have this blog series in which the first post (01_winter-bohemia) is written as a .md file and the second post (02_pseudo-spring-bohemia) is written as .Rmd. The former is rendering with no issues but the latter is not rendering neither in the website nor in the local serve. This is not a problem restricted to "series" posts.

The only lead that I am yet to try myself is this one but I am not able to figure out how to get that folder into my directory. I'd greatly appreciate any help with this, as I intend to use .Rmd majorly in my blog!

yihui commented 2 years ago

@rikudoukarthik I don't see your index.md or index.html output file: https://github.com/rikudoukarthik/rikudou/tree/main/content/blog/09_czech-birding/02_pseudo-spring-bohemia

Also: https://bookdown.org/yihui/blogdown/a-quick-example.html#when-in-doubt-run-blogdowncheck_site

rikudoukarthik commented 2 years ago

I don't see your index.md or index.html output file:

I had index.html earlier but I removed it thinking that might be the reason for the issue.

Also: https://bookdown.org/yihui/blogdown/a-quick-example.html#when-in-doubt-run-blogdowncheck_site

Thanks. I tried blogdown::build_site(build_rmd = "newfile"), as suggested by blogdown::check_site(), but got the following error:

Start building sites … 
hugo v0.91.0-D1DC0E9A+extended windows/amd64 BuildDate=2021-12-17T09:50:20Z VendorInfo=gohugoio
WARN 2022/07/28 08:48:01 Page.UniqueID is deprecated and will be removed in a future release. Use .File.UniqueID
Error: Error building site: failed to render pages: render of "page" failed: "D:\Website\themes\hugo-apero\layouts\_default\single.html:19:33": execute of template failed: template: _default/single.html:21:11: executing "main" at <partial "shared/post-details.html" .>: error calling partial: "D:\Website\themes\hugo-apero\layouts\partials\shared\post-details.html:19:33": execute of template failed: template: partials/shared/post-details.html:19:33: executing "partials/shared/post-details.html" at <.>: range can't iterate over Czech Birding
Total in 1591 ms

When I had blogdown.method = "markdown" in my .Rprofile I noticed the above command produced index.md but then returned this error, while with blogdown.method = "html" it directly returned the error.

Also, is it necessary to run that command for all .Rmd posts?

yihui commented 2 years ago

range can't iterate over Czech Birding

@rikudoukarthik This series field needs to be an array instead of a scalar: https://github.com/rikudoukarthik/rikudou/blob/c318d2873d52546890457bf2ccf281db67836c5f/content/blog/09_czech-birding/_index.md?plain=1#L19

WARN 2022/07/28 08:48:01 Page.UniqueID is deprecated and will be removed in a future release. Use .File.UniqueID

This has been fixed a few months ago in the theme: https://github.com/hugo-apero/hugo-apero/pull/78 You might be using an older version of the theme.

rikudoukarthik commented 2 years ago

This series field needs to be an array instead of a scalar

@yihui Did you mean that I have to enter the series name as points like so:

series:
- Czech Birding

I was mostly using Alison's code as reference, and she has done the same as me here, as a scalar. Please let me know if I have misunderstood. I tried the above as well, but to no avail.

You might be using an older version of the theme.

I now updated the Apero theme but it still gave an error (a different one this time, see below), so I also updated Hugo from 0.91.0 to 0.101.0, after which the check_site() stopped listing in the TODO the unrendered .Rmd file. However, serve_site() is now giving the same error I mentioned above (while it was working without error before updating Hugo). This is the error:

> blogdown::serve_site()
Launching the server via the command:
  C:/Users/Karthik Thrikkadeeri/AppData/Roaming/Hugo/0.101.0/hugo.exe server --bind 127.0.0.1 -p 4321 --themesDir themes -t hugo-apero -D -F --navigateToChanged
Error: Error building site: "D:\Website\content\blog\09_czech-birding\02_pseudo-spring-bohemia\index.html:10:14": failed to render shortcode "blogdown/postref": failed to process shortcode: "D:\Website\layouts\shortcodes\blogdown\postref.html:1:10": execute of template failed: template: shortcodes/blogdown/postref.html:1:10: executing "shortcodes/blogdown/postref.html" at <getenv "BLOGDOWN_POST_RELREF">: error calling getenv: access denied: "BLOGDOWN_POST_RELREF" is not whitelisted in policy "security.funcs.getenv"; the current security configuration is:

[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
yihui commented 2 years ago

However, serve_site() is now giving the same error I mentioned above (while it was working without error before updating Hugo).

It seems you are not using the latest version of blogdown. Please update your R packages, restart R, and serve the site again. You should see layouts/shortcodes/blogdown/postref.html updated, and the error should be gone.

rikudoukarthik commented 2 years ago

Thank you so much @yihui! My bad for not noticing that earlier. I appreciate you taking time out for such a basic issue. I love your packages and will keep exploring!