Closed carstenbauer closed 3 years ago
Turns out that we also have to tell the julia/juliacon blocks (pythontex) to start julia with --project. How do we do that?
We solve this problem by having pull_julia_code.jl
export to a Julia package instead, and we import that package. That package has its own requirements .toml that allows us to manage dependency installation.
I don't know how to change the pythontex startup command. Likely a change to pythontex itself.
If you're going to work on this, can you also update the Gitlab CI template accordingly?
If you're going to work on this, can you also update the Gitlab CI template accordingly?
Done. I also updated the README accordingly. There is no need to use jlpkg anymore.
We solve this problem by having
pull_julia_code.jl
export to a Julia package instead, and we import that package. That package has its own requirements .toml that allows us to manage dependency installation.
I'm not sure I understand this fully. Even if you make pull_julia_code.jl
export to a package (this part isn't in the repo is it?), what do you mean by "import it"? Wouldn't you still have to add this package to the global env for pythontex to find it? After all, it's the purpose of this PR to make the repo a self-contained Julia project such that no packages have to be added to the global (v1.x) environment.
Note to future self:
Apparently only the juliaconsole
blocks aren't executed in the local env. The juliacode
blocks seem to work fine. However I haven't really understood why ...
Trying to investigate this I found https://github.com/gpoore/pythontex/blob/master/pythontex/pythontex_engines.py. Here, both julia
and juliacon
CodeEngine
s are being defined. Note that juliacon
doesn't seem to include the julia template nor the "julia wrapper".
Even if you make
pull_julia_code.jl
export to a package (this part isn't in the repo is it?), what do you mean by "import it"? Wouldn't you still have to add this package to the global env for pythontex to find it?
Basically, we have a Julia package that loads all_algorithm_blocks.jl
. That part is not in the repo. We run using OurPackage
within a pythontex call at the top of book.tex
. Said package has its own github repo and .toml such that we can update its dependencies.
\begin{jlcode}
using OurPackage
\end{jlcode}
We actually moved the code to its own repo to benefit from precompilation. We were running into huge load times. Our new book has more, and heftier, code than the first textbook. The sacrifice we made is that now there are two repos - one for the book and one for the Julia code associated with the book. The latter is only updated when we add new code utilities or code dependencies.
After all, it's the purpose of this PR to make the repo a self-contained Julia project such that no packages have to be added to the global (v1.x) environment.
The package-export approach does store all dependencies in one place. At the same time, we are not running ] instantiate
as you would with this standalone-approach. This PR may very well be the way to go.
Turns out that we also have to tell the julia/juliacon blocks (pythontex) to start julia with --project. How do we do that?
You can second my PR here
It got merged 🎉
Is this PR good to go?
I don't think so. I haven't finished it based on @BeastyBlacksmith PR mentioned above. Will try to wrap things up but (unfortunately) can't promise anything.
Any update on this?
No, but I will give this half an hour today. Let's see if this is enough to finish things.
They haven't tagged a new release yet. Therefore, despite the PR being merged, the project flag is still missing in pythontex 0.17. I asked whether they could tag a new release in https://github.com/gpoore/pythontex/pull/158. After all the last release is from 2019.
It looks like it was tagged. Shall we merge this?
I'll take a look at this tomorrow and if things work I'll let you know and your can merge.
This could, in principle, work now. However, I'm not able to compile the book because I'm using 1.6 and probably too recent package versions (that didn't exist back when the book was written), e.g. https://github.com/sisl/tufte_algorithms_book/issues/31 and https://github.com/sisl/tufte_algorithms_book/issues/30. Would be great if someone could tell me the old package version numbers.
Alright, after downgrading Pygments to version 2.6.1, using Weave v0.10.2, and using julia 1.1.1 things seem to work fine. I comitted the Manifest.toml so that people always get the same package versions. (Of course, the can always update packages if they like.)
Would be great if someone else could test this. Otherwise, this can be merged IMO.
I just updated the GitLab CI and it works perfectly on Julia 1.0-1.2. https://gitlab.com/johnnychen94/tufte_algorithms_book/-/pipelines/336146017
And submitted PR #32 to test with Julia 1.x.
Drop the old REQUIRE file in favor of the new Project.toml. This way the repository (up to a Manifest.toml) forms a Julia project and can easily be
] instantiate
d.I also adjusted the makefile such that julia is started with
--project
, i.e. in the local environment.Remaining question: should we also specify compatibilities through
[compat]
entries?Closes #23.