quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.95k stars 326 forks source link

Installing via Conda #61

Closed tomkom closed 1 year ago

tomkom commented 3 years ago

Dear all, I have just discovered Quarto and I am very excited by it, in particular as a way to enable students to render well formatted reports directly from notebooks, and as a way to work on scientific papers with a relatively simple way to pass journal templates to the project for rendering ( something to explore).

I am wondering whether it is possible to make sure the python environment used ( conda) is the same one where quarto executes, by having quarto install from conda ( forge). Do you plan to release under conda forge? Or is there a way to specify this, for instance through the nested pip install from github (as in https://stackoverflow.com/questions/19042389/conda-installing-upgrading-directly-from-github) .

Thanks, Martin

jjallaire commented 3 years ago

Currently Quarto has very minimal Python dependencies (just jupyter) and can work with pretty much any version of Python (Conda or otherwise) that it is run within. So if you arrange your shell to have the requisite Conda environment active when you invoke quarto then all will be well.

One point of clarification: while Quarto has Python code in it for executing notebooks, it is a standalone executable mostly written in Typescript (w/ the deno runtime) so I'm not sure if we could distribute via conda-forge even if we wanted to.

tomkom commented 3 years ago

Thank you for the speedy response!

  1. Dependencies - excellent. But to clarify do you then need to install pandoc separately?
  2. Conda: conda is a "general purpose packaging system", for any binaries (AFAIK, no expert). Certainly, you can find recipes to install all sorts of stuff ( incl Latex, but also TypeScript, or even large applications, such as QGIS or Inkscape). See here for the TypeScript feedstock https://github.com/conda-forge/typescript-feedstock
  3. As quarto needs to be installed from a binary, this would make it absolutely doable for, e.g., low-privilege users ( such as in a shared cloud environment running conda) to install it, and run, possibly as part of a scripted environment with all dependencies ( incl pandocs etc), all without sudo. This would also allow to transfer over well defined projects, making sure that any user would have the full recipe for the project, including a quarto environment.
jjallaire commented 3 years ago

1) They do not need to install pandoc (it comes with quarto)

2) Understood re: Conda being quite general. I don't see a way though to install Deno yet (seems like this would be a natural though). Note we also include dart sass and esbuild (neither of which I see on conda-forge). So I think we'd need a minimum of 4 new conda-forge packages (including Quarto). Further, Quarto is currently tracking quite closely to the latest versions of Deno and Pandoc (sometimes even including development versions). In summary I think we definitely could be on conda-forge but it would require a few new packages and perhaps might not be the best idea yet given how particular our dependencies can be. Perhaps something to revisit in a few months time.

3) Very good point about low-priv users! Currently our Windows and Mac installers do work for low-priv users but Linux .deb doesn't. In the short term we will create a tarball installer for Linux that you can just extract anywhere (cc @dragonstyle). Longer term though I agree that conda-forge would be a great solution to this problem.

dragonstyle commented 3 years ago

Followup on tarball installer- we are already building these as a part of our releases (the quarto-<version>.tar.gz is the ready to extract version).

https://github.com/quarto-dev/quarto-cli/releases/download/v0.2.28/quarto-0.2.28.tar.gz

The quarto binary is located in the bin subdirectory - make this directory accessible via path or call directly and it should work well.

If you’d like to give it a try and have any feedback, be happy to hear your thoughts!

tomkom commented 3 years ago

Thanks a lot for replies, will keep tracking re Conda. Dug a bit into the js world, had no idea about Deno et al. It would likely require some more digging into conda, but should be possible ( but beyond my knowledge). Will explore the installfrom the tar.gz versions for now, although that will require sudo I think.

jjallaire commented 3 years ago

It shouldn't require sudo (or at least it's not at all intended to!). You should be able to extract that anywhere and then add the bin dir to the PATH.

tomkom commented 3 years ago

So, trouble. Trying the binary build (from the tar.gz). I am getting a dependency issue ( and I have no idea about this library). Note, this is on a server, trying to reproduce the env that my low priv users would have:

$ cd testquarto/ $ tar -xf quarto-0.2.28.tar.gz $ ./bin/quarto /testquarto/bin/deno: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /testquarto/bin/deno).

Martin

jjallaire commented 3 years ago

What version of Linux are you running on the server? If it's RHEL or CentOS 7 it looks like Deno doesn't currently support that version: https://stackoverflow.com/questions/55363823/redhat-centos-glibc-2-18-not-found (although it seems like many people think they should and some work/discussion is underway).

jjallaire commented 3 years ago

Relevant SO thread that discusses the library dependency issue: https://stackoverflow.com/questions/55363823/redhat-centos-glibc-2-18-not-found

jjallaire commented 3 years ago

Here's the Deno issue tracking this: https://github.com/denoland/deno/issues/1658

tomkom commented 3 years ago

I have no idea (still ) what deno is :), and I am not admin on this server, but will pass the info. Server info is: NAME="Red Hat Enterprise Linux Server" VERSION="7.9 (Maipo)"

And I have found that people refer to this issue also with respect to Python version (3.6), I tested this when the active binary was Pythin 3.8.5 ( should not matter).

I can confirm the GLIB installed is 2.17 $ldd --version ldd (GNU libc) 2.17

If RHL or CentOS are not supported, I suggest to at least note that in the install instructions/reqs. Will explore what upgrades my admin can do, and report.

jjallaire commented 3 years ago

@dragonstyle Is there a way to add a note about this to the releases page?

@tomkom Deno is a JavaScript/TypeScript runtime: https://deno.land/

dragonstyle commented 3 years ago

Yep, will do.

jjallaire commented 3 years ago

Note that RHEL/CentOS version 8 are supported by Deno.

statquant commented 3 years ago

Hello, I came here trying to install quarto on our centos 7. I understand that centos 7 won't be supported, I will try it when we migrate to RHEL 8. Thanks for quarto

jgunstone commented 3 years ago

this looks like a really amazing tool. I've been turning Juptyer Notebooks into markdown files (with jupytext) and then using pandoc to convert to docx for a while: quarto looks like the perfect formalization of that workflow (and much more!). Thank you!

currently the installation method is unclear to me; having a simple to install conda package would lower the barrier for immediate use for me and i'm sure many other users.

@jjallaire

3. Longer term though I agree that conda-forge would be a great solution to this problem.

^ couldn't agree more!

To give a use case; I build Voila Apps in python that are deployed using Binder / Repo2docker. These tools create docker images with environments that are created based on a environment.yml file. The ability to simply add quarto to the environment.yml file would simply enable me to easily use quarto to convert the Voila Jupyter Notebook App into a docx or pdf output.

many thanks for the great work

jjallaire commented 3 years ago

Agree 100% that it would be great to have this on conda-forge!

The thing we have to surmount here is that Quarto embeds several other non-trivial binaries (only one of which I can see is currently packaged for conda-forge -- pandoc itself). Here are our dependencies (and their implementation languages):

Note that pandoc is available on conda-forge here (https://github.com/conda-forge/pandoc-feedstock) as a set of repackaged binaries (not built from Haskell source). I believe that in order to get Quarto onto conda-forge we'd need repackaged binaries of Deno, Dart Saas, and esbuild (or perhaps conda-forge can build one or all of Rust, Dart, and Go binaries?)

Anyway, we don't have direct experience with Conda packaging on our team so we would need help with this effort.

chlazaris commented 2 years ago

Quarto looks like an excellent tool. Very promising indeed! The problem is that as mentioned here (e.g., https://github.com/quarto-dev/quarto-cli/issues/61#issuecomment-887110934), we still use CentOS 7 and we have the aforementioned issues with Demo and GLIB. Are there any workarounds at the moment? Also, is the tool going to be available on Conda channels anytime soon? Thank you!

lachlansimpson commented 2 years ago

Ticket: #78264 Ticket: #78297

msarahan commented 2 years ago

If you have Quarto 1.1.46 with https://github.com/quarto-dev/quarto-cli/commit/4d687f8b6edbe35d6e2431c2e6c0f1a815fd4951 or newer (which you could get from installing from source, not from an official release), then you can install the conda-forge deno package:

conda install -c conda-forge deno=1.22.0

There are other deno versions available there, but they are not tested with Quarto and your mileage may vary.

Next, set the QUARTO_DENO environment variable (assuming your conda environment is active/on PATH):

export QUARTO_DENO=$(which deno)

Run Quarto - you should be using the conda-forge build of deno now, which is built against an older glibc.

More official support is coming soon. Once https://github.com/quarto-dev/quarto-cli/pull/1687 is available in a tag of Quarto, I will make conda packages for Quarto on conda-forge.

msarahan commented 2 years ago

One other hack that you can do today if you can't wait for the next release: you can replace the deno executable that ships with quarto with the conda one instead. You can download the package directly from https://anaconda.org/conda-forge/deno/1.22.0/download/linux-64/deno-1.22.0-hf47965d_0.tar.bz2. Replace the deno executable in your quarto folder (in bin/tools) with the executable from the bin folder in the conda package.

jthomasmock commented 1 year ago

Given that a few folks have come to this thread for more info re: Deno + RHEL7/CentOS7 - the Quarto team has a specific install for that OS now.

Direct link to Quarto 1.2 compatible with RHEL/CentOS7: https://github.com/quarto-dev/quarto-cli/releases/download/v1.2.269/quarto-1.2.269-linux-rhel7-amd64.tar.gz

Full download page at: https://quarto.org/docs/download/ (see the RHEL7 Tarball)

cderv commented 1 year ago

And thanks to @msarahan , windows build should be available now on conda-forge https://anaconda.org/conda-forge/quarto

@msarahan we are all good know right ? We can close ?

juanfcocontreras commented 1 year ago

Would it be possible to generate a version for Macs with arm processor before closing this issue?

Thanks!

msarahan commented 1 year ago

osx-arm64 packages are available now

juanfcocontreras commented 1 year ago

osx-arm64 packages are available now

Installed without any issue! Thank you so much!!

allenmanning commented 1 year ago

It seems like we can close.

kaedonkers commented 1 year ago

Great to see Quarto on conda-forge! However, it took some digging to find this issue and determine that it was indeed available on conda because this info is not included in the Getting Started page of the Quarto docs. Could a section about installing Quarto via conda be added to the docs please?

cderv commented 1 year ago

@cwickham I believe this would also be a topic for https://github.com/quarto-dev/quarto-cli/issues/5047