search

cyhsieh-psy / SMLP_exercise

0 stars 0 forks source link

Rendering the Quarto file #3

Open dmbates opened 1 year ago

dmbates commented 1 year ago

I have added some files in branch called db/quarto

If you have julia 1.9 (the current release version) installed you can render the .qmd document after installing the IJulia package. The sequence would be

% git clone https://github.com/cyhsieh-psy/SMLP_exercise
Cloning into 'SMLP_exercise'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 28 (delta 7), reused 6 (delta 0), pack-reused 0
Receiving objects: 100% (28/28), 1.18 MiB | 4.26 MiB/s, done.
Resolving deltas: 100% (7/7), done.
% cd SMLP_exercise 
% git checkout -b db/quarto origin/db/quarto
branch 'db/quarto' set up to track 'origin/db/quarto'.
Switched to a new branch 'db/quarto'

% julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.3 (2023-08-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Pkg; Pkg.instantiate()
    Updating registry at `~/.julia/registries/General.toml`
    Updating `/private/tmp/SMLP_exercise/Project.toml`
  [cbdf2221] + AlgebraOfGraphics v0.6.16
  [69666777] + Arrow v2.6.2
  [336ed68f] + CSV v0.10.11
  [13f3f980] + CairoMakie v0.10.8
  [a93c6f00] + DataFrames v1.6.1
  [8197267c] + IntervalSets v0.7.7
  [ff71e718] + MixedModels v4.21.0
  [b12ae82c] + MixedModelsMakie v0.3.26
  [92933f4c] + ProgressMeter v1.8.0
  [6f49c342] + RCall v0.13.17
  [5064a6a7] + StandardizedPredictors v1.0.1
  [2913bbd2] + StatsBase v0.34.0
  [9d95f2ec] + TypedTables v1.4.3
    Updating `/private/tmp/SMLP_exercise/Manifest.toml`
  [621f4979] + AbstractFFTs v1.5.0
  [398f06c4] + AbstractLattices v0.2.1

  *many, many more lines of output*

  [10745b16] + Statistics v1.9.0
  [4607b0f0] + SuiteSparse
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.5+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.2+0
  [14a3606d] + MozillaCACerts_jll v2022.10.11
  [4536629a] + OpenBLAS_jll v0.3.21+4
  [05823500] + OpenLibm_jll v0.8.1+0
  [efcefdf7] + PCRE2_jll v10.42.0+0
  [bea87d4a] + SuiteSparse_jll v5.10.1+6
  [83775a58] + Zlib_jll v1.2.13+0
  [8e850b90] + libblastrampoline_jll v5.8.0+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
        Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`

julia>  exit()

%

At this point you have the environment set up but you still need to install the IJulia package, usually in the global environment, not in the current project. Start julia without the --project flag and execute

using Pkg; Pkg.install("IJulia")

Finally install quarto as described at https://quarto.org/docs/get-started/ and run quarto check which should produce a result like

% quarto check

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.1: OK
      Dart Sass version 1.55.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.3.450
      Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.12 (Conda)
      Path: /usr/local/bin/python3
      Jupyter: 5.3.1
      Kernels: julia-1.9, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.3.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library
      knitr: 1.43
      rmarkdown: 2.24

[✓] Checking Knitr engine render......OK

In particular, you want to ensure that julia-1.9 is listed as one of the kernels in the block about Python 3 and Jupyter.

Then you could try

% quarto render CL2arrow.qmd --to html

Starting julia-1.9 kernel...Done

Executing 'CL2arrow.ipynb'
  Cell 1/13...Done
  Cell 2/13...Done
  Cell 3/13...Done
  Cell 4/13...Done
  Cell 5/13...Done
  Cell 6/13...Done
  Cell 7/13...Done
  Cell 8/13...Done
  Cell 9/13...Done
  Cell 10/13...Done
  Cell 11/13...Done
  Cell 12/13...Done
  Cell 13/13...Done

pandoc 
  to: html
  output-file: CL2arrow.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Creating an Arrow IPC file from the Chinese Lexicon data
  author: Douglas Bates
  jupyter: julia-1.9
  bibliography:
    - CLS.bib

Output created: CL2arrow.html

If you substitute preview for render in the call to quarto it will open the HTML document in a web browser.

If this succeeds you can edit the file CL2arrow.qmd in VSCode, after installing the Julia and Quarto extensions, or in the RStudio IDE. As far as I can tell the "render" button works in RStudio, because it just calls quarto render in a Terminal window, but it doesn't open a preview. Also, execution of individual code blocks doesn't work (as far as I can tell) because the IDE is not communicating with a Julia instance, just with R.

Let me know if you try this and, if so, does it work?

kliegl commented 1 year ago

This is great! Thank you.

cyhsieh-psy commented 1 year ago

Hello,

Sorry for the last minute call. It does not work quite well. It seems that there is a problem with Python and/or Jupyter. However, there's no problem for me to open the file CL2arrow.qmd in VSCode. (BTW I use MacBook M1)

Here is my quarto check:

quarto check

[✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.1: OK Dart Sass version 1.55.0: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.3.450 Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK Version: 3.9.10 (Conda) Path: /Users/cyhsieh/miniforge3/bin/python Jupyter: 5.3.1 Kernels: julia-1.9, python3

(/) Checking Jupyter engine render....Traceback (most recent call last): File "/Users/cyhsieh/miniforge3/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/Users/cyhsieh/miniforge3/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel_launcher.py", line 15, in from ipykernel import kernelapp as app File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 52, in from .ipkernel import IPythonKernel File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 20, in from .comm.comm import BaseComm File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel/comm/init.py", line 3, in from .comm import Comm File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel/comm/comm.py", line 15, in from ipykernel.kernelbase import Kernel File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 38, in import psutil File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/init.py", line 123, in from . import _psosx as _psplatform File "/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/_psosx.py", line 14, in from . import _psutil_osx as cext ImportError: dlopen(/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so, 0x0002): tried: '/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (no such file), '/Users/cyhsieh/miniforge3/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) 2023-09-08 22:02:23,309 - traitlets - ERROR - Error occurred while starting new kernel client for kernel 1af016aa-2c26-47c0-a2e6-13b841e79bc6: Kernel died before replying to kernel_info

Kernel died before replying to kernel_info [✓] Checking Jupyter engine render....OK

And here is my quarto preview CL2arrow.qmd --to html:

quarto preview CL2arrow.qmd --to html

Starting julia-1.9 kernel...Done

Executing 'CL2arrow.ipynb' Cell 1/13...Done Cell 2/13...Done Cell 3/13...Done Cell 4/13...Done Cell 5/13...Done Cell 6/13...ERROR:

An error occurred while executing the following cell:

R""" dat <- arrow::read_ipc_file($arrowfn) # pass the file name to R through string interpolation tibble::glimpse(dat) """

REvalError: loadNamespace(x) でエラー: ‘arrow’ という名前のパッケージはありません 呼び出し: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart

Stacktrace: [1] handle_eval_stderr(; as_warning::Bool) @ RCall ~/.julia/packages/RCall/aK5sD/src/io.jl:174 [2] handle_eval_stderr @ ~/.julia/packages/RCall/aK5sD/src/io.jl:168 [inlined] [3] reval_p(expr::Ptr{LangSxp}, env::Ptr{EnvSxp}) @ RCall ~/.julia/packages/RCall/aK5sD/src/eval.jl:103 [4] reval_p(expr::Ptr{RCall.ExprSxp}, env::Ptr{EnvSxp}) @ RCall ~/.julia/packages/RCall/aK5sD/src/eval.jl:119 [5] reval(str::String, env::RObject{EnvSxp}) @ RCall ~/.julia/packages/RCall/aK5sD/src/eval.jl:136 [6] top-level scope @ ~/.julia/packages/RCall/aK5sD/src/macros.jl:67

cyhsieh-psy commented 1 year ago

I'm not sure if this helps. So as I uninstall Jupyter, I rerun quarto check and get this. It seems that Julia is not listed as one of the kernels in this case:

quarto check

[✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.1: OK Dart Sass version 1.55.0: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.3.450 Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK Version: 3.9.10 (Conda) Path: /Users/cyhsieh/miniforge3/bin/python Jupyter: (None)

  Jupyter is not available in this Python installation.
  Install with conda install jupyter

[✓] Checking R installation...........OK Version: 4.1.1 Path: /Library/Frameworks/R.framework/Resources LibPaths:

  • /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library knitr: 1.43 rmarkdown: 2.22

[✓] Checking Knitr engine render......OK

dmbates commented 1 year ago

This is an unfortunately common situation. Many operating systems install a version of Python which may or may not have a version of Conda for package installation and maintenance. To avoid the problems of tracking down other installed versions of Python and/or Conda, the PyCall.jl package, by default, installs a version in ~/.julia/conda/3/bin/ To use quarto to render Julia-based documents you need to ensure that this is the version accessed by quarto. I did it by adding a symbolic link in /usr/local/bin/ but I'm not sure if that is the recommended approach.

@palday Do you have a recommended approach for getting the right Python? Quarto apparently looks for an environment variable QUARTO_PYTHON (see https://quarto.org/docs/projects/environment.html). It looks as if @cyhsieh-psy is on a Mac.

cyhsieh-psy commented 1 year ago

Below is my hardware overview:

system_profiler SPHardwareDataType Hardware:

Hardware Overview:

  Model Name: MacBook Air
  Model Identifier: MacBookAir10,1
  Model Number: MGN73TA/A
  Chip: Unknown
  Total Number of Cores: 8 (4 performance and 4 efficiency)
  Memory: 8 GB
  System Firmware Version: 8422.141.2
  OS Loader Version: 8422.141.2
  Serial Number (system): FVFG919XQ6L5
  Hardware UUID: 846C34AD-C37D-58B4-9D45-1A796D9FBC0A
  Provisioning UDID: 846C34AD-C37D-58B4-9D45-1A796D9FBC0A
  Activation Lock Status: Enabled
palday commented 1 year ago

@cyhsieh-psy If you can go back to your original setup, I think that's the best solution. That one file is failing to render because it has some inline R code and uses RCall to execute it. You need to make sure that RCall is installed in the appropriate Julia environment.

For the most recent issue (uninstalled jupyter), there are two possible fixes here, let's see which is easiest. :wink:

Use the IJulia-maintained hidden Mini-Conda

(I think this is the better way, if it works). Basically, you need to specify which Python to grab and as @dmbates mentioned, you can do this with QUARTO_PYTHON.

% QUARTO_PYTHON=~/.julia/conda/3/bin/python quarto check

and if that seems happy...

% QUARTO_PYTHON=~/.julia/conda/3/bin/python quarto render

Install jupyter in your existing system conda installation

%  /Users/cyhsieh/miniforge3/bin/conda install jupyter

and after that completes....

% QUARTO_PYTHON=~/.julia/conda/3/bin/python quarto check

and if that seems happy...

% QUARTO_PYTHON=~/.julia/conda/3/bin/python quarto render

At this point, you might run into an error about not finding an appropriate Julia kernel, but we can cross that bridge when we get there.

cyhsieh-psy commented 1 year ago

Hi

I guess I'm in the right direction: it's all about the path, the directory or whatnot. The directory of python under /.julia/ folder doesn't look straightforward. I run the code below anyway but fail to find both jupyter and julia. Also I cannot install jupyter under this directory. I'm not sure if it's a right thing to do. Sorry to annoy you but I think I'm almost there ...

QUARTO_PYTHON=/Users/cyhsieh/.julia/conda/3/aarch64/bin/python quarto check

[✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.1: OK Dart Sass version 1.55.0: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.3.450 Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK Version: 3.10.12 (Conda) Path: /Users/cyhsieh/.julia/conda/3/aarch64/bin/python Jupyter: (None)

  Jupyter is not available in this Python installation.
  Install with conda install jupyter

[✓] Checking R installation...........OK Version: 4.1.1 Path: /Library/Frameworks/R.framework/Resources LibPaths:

  • /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library knitr: 1.43 rmarkdown: 2.22

[✓] Checking Knitr engine render......OK

palday commented 1 year ago

@cyhsieh-psy does everything work now?