update sagetex to version 3.6.1

[dimpase]

Previous update: #30342, merged 2020-08-20.

Depends on #34219

CC: @kcrisman @mkoeppe @orlitzky @slel @jhpalmieri @EmmanuelCharpentier

Component: packages: standard

Author: Dima Pasechnik, Matthias Koeppe

Branch/Commit: add10cd

Reviewer: Emmanuel Charpentier

Issue created by migration from https://trac.sagemath.org/ticket/32887

[dimpase]

Author: Dima Pasechnik

[dimpase]

New commits:

9e26d0a

update sagetex to version 3.6

[dimpase]

Commit: 9e26d0a

[dimpase]

Branch: u/dimpase/packages/sagetex/v3.6

[mkoeppe]

comment:2

sagetex has a new dependency, pyparsing, that also needs to be added as dependency of the SPKG. - https://github.com/sagemath/sagetex/blob/master/setup.py#L16

And the 3.6 release is defective - it still identifies itself as 3.5 - https://github.com/sagemath/sagetex/blob/master/setup.py#L7

[mkoeppe]

comment:3

Or, rather, what is on https://github.com/sagemath/sagetex/ does not match what is packaged up in the release tarball

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

180f8a1

correct version in setup.py, and deps

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 9e26d0a to 180f8a1

[dimpase]

comment:5

should all be fixed now, sorry. To much different metadata to tie up manually here...

[dimpase]

comment:6

I think I'll eventually add a small configure script to set up all the versions right. If only there was a way to put tarballs up on GitHub and creating GitHub releases from the command line.

[mkoeppe]

comment:7

What is the tarball doing on GitHub? Is it not supposed to be on CTAN (https://ctan.org/pkg/sagetex?lang=en) or on PyPI?

[dimpase]

comment:8

Replying to @mkoeppe:

What is the tarball doing on GitHub? Is it not supposed to be on CTAN (https://ctan.org/pkg/sagetex?lang=en) or on PyPI?

there is something called sagetex.zip on CTAN - but the process of publishing there is manual, and I have not put this there yet.

And I don't know anything about PyPI publishing.

[kcrisman]

comment:9

Is it not supposed to be on CTAN (https://ctan.org/pkg/sagetex?lang=en) or on PyPI?

Unless something has changed, we do not want it on CTAN, because that will not match versions with Sage. Dan always made sure that the versions matched (indeed, I think this is in the release instructions for SageTeX?) and so one has to manually move the .sty file, or did have to do.

[dimpase]

comment:10

Hmm, what do you mean by "we don't want it on CTAN"? Have you forgotten how we started putting in on CTAN few years ago? (And I did it for versions 3.4 and 3.5, and will do for 3.6)

[saraedum]

comment:11

Replying to @dimpase:

I think I'll eventually add a small configure script to set up all the versions right. If only there was a way to put tarballs up on GitHub and creating GitHub releases from the command line.

Have you looked into rever? I use it to create tarballs for autoconfiscated packages, release them on GitHub and on PyPI. See, e.g., https://github.com/flatsurf/flatsurf/blob/master/rever.xsh.

[kcrisman]

comment:12

Unless something has changed,

Apparently it has.

Hmm, what do you mean by "we don't want it on CTAN"? Have you forgotten how we started putting in on CTAN few years ago? (And I did it for versions 3.4 and 3.5, and will do for 3.6)

That is good news, and I hope it will keep track with the Sage versions well.

[dimpase]

comment:13

Replying to @kcrisman:

Unless something has changed,

Apparently it has.

Hmm, what do you mean by "we don't want it on CTAN"? Have you forgotten how we started putting in on CTAN few years ago? (And I did it for versions 3.4 and 3.5, and will do for 3.6)

That is good news, and I hope it will keep track with the Sage versions well.

Karl, it's almost 2 years old news:

Date: Thu, 24 Jan 2019 18:50:56 +0000
References: <CAKqbAeFMwHMHYbOfA7SOXOJuuv6aBU=89R0Cz8iddJxkmg+64g@mail.gmail.com>
In-Reply-To: <CAKqbAeFMwHMHYbOfA7SOXOJuuv6aBU=89R0Cz8iddJxkmg+64g@mail.gmail.com>
Message-ID: <CAAWYfq211+bChdmArcJAPwK62opKnSxEfERmtS_xBbcV3kEH_A@mail.gmail.com>
Subject: Re: are you guys the CTAN maintainers for SageTeX now?
From: Dima Pasechnik <...>
To: Dan Drake <...>
Cc: Karl-Dieter Crisman <...>
Content-Type: text/plain; charset="UTF-8"

Hi Dan,
the update is in! cf https://ctan.org/pkg/sagetex?lang=en
Cheers,
Dima

On Wed, Jan 23, 2019 at 1:45 AM Dan Drake <> wrote:
>
> Hi Karl-Dieter, Dima,
>
> Following up -- I see the reply from the CTAN guy. Are you guys now considered the CTAN maintainers for SageTeX? Can you update or remove it?
>
> --
> Ceci n'est pas une .signature.

[kcrisman]

comment:14

Karl, it's almost 2 years old news:

I give up. I clearly can no longer keep 'Trac'.

[dimpase]

comment:15

Replying to @dimpase:

Replying to @mkoeppe:

What is the tarball doing on GitHub? Is it not supposed to be on CTAN (https://ctan.org/pkg/sagetex?lang=en) or on PyPI?

there is something called sagetex.zip on CTAN - but the process of publishing there is manual, and I have not put this there yet.

And I don't know anything about PyPI publishing.

The package's Python code (apart from setup.py) is generated by TeX from a .dtx file. Anyhow, I tried uploading its Makefile-created tarball (one used by Sage) anf got

$ twine upload  dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading sagetex-3.6.tar.gz
100%|████████████████████████| 71.0k/71.0k [00:00<00:00, 73.0kB/s]
Error during upload. Retry with the --verbose option for more details.
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
The name 'sagetex' isn't allowed. See https://pypi.org/help/#project-name for more information.

Perhaps some metadata is missing, I don't know. There is no sagetex project on PyPI as far as I can tell.

Let's get this in as is, and deal with a better distribution mechanism later.

[orlitzky]

comment:17

Is sagetex actually used by sage? TBH I've managed to avoid it until now.

[dimpase]

comment:18

No, sagetex is a !Sage/Latex package. It's hard to spin it off, cause as a Python package, almost all of it is generated by running TeX on a .dtx file.

[orlitzky]

comment:19

Ok, I see what's happening. Sage needs the python bits installed before the user can use the latex package. Earlier I was wondering why we install this into sage at all, as opposed to just installing it as a latex package. Still, I think it should be optional.

I'll test it out when I get a chance.

[mkoeppe]

comment:20

This version is still not on CTAN.

[dimpase]

comment:21

We do it in lockstep. I don't want to put anything on CTAN ahead of the update here.

(CTAN updates are pretty quick)

[mkoeppe]

comment:22

Does sagetex do anything that depends on the version of Sage?

As far as I can see, it imports nothing from the Sage library and only runs whatever sage can be found on PATH (or what is provided using the --sage option).

[dimpase]

comment:23

Itself it does not, but examples and docs do import stuff from Sage. Also, there is a version check: see #8035 (Sage "knows" the version of sagetex it should be using). Maybe it's obsolete.

[kcrisman]

comment:24

Itself it does not, but examples and docs do import stuff from Sage. Also, there is a version check: see #8035 (Sage "knows" the version of sagetex it should be using).

That was what I referred to above.

Maybe it's obsolete.

That is possible; no one has checked that out recently. It would definitely be worth knowing if we could get rid of that requirement (see point 2 in HACKING.md).

The idea (Dan called it "one of the biggest problems with SageTeX") was that if you have the wrong style file, bad things might happen. Presumably some of those bad things would be different behavior of certain (ST) commands, or some not working at all? Cc:ing John, who originally suggested this here a dozen years ago; I never had that experience personally, but I don't use all the advanced capabilities of ST either.

[jhpalmieri]

comment:25

By the way, the information in https://doc.sagemath.org/html/en/tutorial/sagetex.html about the location of sagetex is out of date: it should say SAGEROOT/venv/share/texmf/tex/latex/sagetex/, and this occurs in several places.

[jhpalmieri]

Changed branch from u/dimpase/packages/sagetex/v3.6 to u/jhpalmieri/packages/sagetex/v3.6

[jhpalmieri]

Changed commit from 180f8a1 to 48f33a2

[jhpalmieri]

comment:27

I don't have time to think about any of this deeply, but here are changes to the location of the sagetex installation in the Sage tree. git grep "local/share.*sagetex" . found no more hits.

---

New commits:

48f33a2

trac 32887: change location of sagetex to venv/share/texmf/...

[jhpalmieri]

comment:28

With both the old and the new versions of sagetex, ./sage -f -c sagetex fails the testsuite. Log uploaded.

[jhpalmieri]

Attachment: sagetex-3.6.log

[dimpase]

comment:29

Replying to @jhpalmieri:

With both the old and the new versions of sagetex, ./sage -f -c sagetex fails the testsuite. Log uploaded.

You are running a development version of TeX:

pdfTeX 3.141592653-2.6-1.40.23 (TeX Live 2022/dev)
...
LaTeX2e <2021-11-15>
L3 programming layer <2021-11-12>
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/article.cls
...

I can't reproduce your error (which is probably about broken/improved(?) TeXLive xcolor package):

[11] (/usr/local/texlive/2021/texmf-dist/tex/latex/listings/lstlang1.sty)
(./sagetex.sagetex.scmd
consecutive:

! Package xcolor Error: Undefined color `dbluecolor'.

See the xcolor package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

I'm running a 2020/21 stable version, and where you got your error I see warnings like

Package xcolor Warning: Incompatible color definition on input line 3.
...

These color definitions there are very old, from 2010 or so.

Perhaps something has changed in xcolor. I'll get back to this tomorrow...

[dimpase]

Work Issues: local->venv in the spkg

[dimpase]

comment:30

Replying to @jhpalmieri:

I don't have time to think about any of this deeply, but here are changes to the location of the sagetex installation in the Sage tree. git grep "local/share.*sagetex" . found no more hits.

---

New commits:

48f33a2

trac 32887: change location of sagetex to venv/share/texmf/...

Oh, OK, there is at least one such place in the package itself, should be fixed too.

[orlitzky]

comment:31

Instead of linking to a private path that will change every once in a while, maybe we should just link to the CTAN for the PDF docs?

If we're uploading to CTAN again, is the advice at the end of the tutorial (avoiding the texlive copy of sagetex) obsolete now? The suggested method of making sagetex visible to latex also gives remarkably bad advice, but I suppose we don't have to fix all of this stuff before doing the upgrade.

[mkoeppe]

comment:32

For reference, https://tug.org/texlive/pkgcontrib.html#exec gives guidance how to prepare a TeX package with executable scripts.

[jhpalmieri]

comment:33

Replying to @dimpase:

Replying to @jhpalmieri:

With both the old and the new versions of sagetex, ./sage -f -c sagetex fails the testsuite. Log uploaded.

You are running a development version of TeX:

Strange, I wonder why. On another machine, tests pass.

[dimpase]

comment:34

the color warning vs error in comment:29 comes from xcolor (the warning, for sure). xcolor manual says:

Note that whenever a color is used that has been defined via color’s \definecolor
command rather than xcolor’s new \definecolor and friends, a warning message
‘Incompatible color definition’ will be issued.

As tikz loads xcolor anyway, perhaps we should just use it.

[dimpase]

comment:35

The following patch to sagetex removes the warning for me (checked with the current xcolor version 2021/10/31, as well as the previous one, from 2016).

John, does it fix the error on your development TeXLive machine?

--- a/py-and-sty.dtx
+++ b/py-and-sty.dtx
@@ -29,7 +29,7 @@
 % and |listings| for the |sagecommandline| environment.
 %    \begin{macrocode}
 \RequirePackage{listings}
-\RequirePackage{color}
+\RequirePackage{xcolor}
 \lstdefinelanguage{Sage}[]{Python}
   {morekeywords={False,sage,True},sensitive=true}
 \lstdefinelanguage{SageOutput}[]{}
@@ -68,9 +68,9 @@
 \lstdefinestyle{SageOutput}{
   style=DefaultSageOutput,
 }
-\definecolor{dbluecolor}{rgb}{0.01,0.02,0.7}
-\definecolor{dgreencolor}{rgb}{0.2,0.4,0.0}
-\definecolor{dgraycolor}{rgb}{0.30,0.3,0.30}
+\providecolor{dbluecolor}{rgb}{0.01,0.02,0.7}
+\providecolor{dgreencolor}{rgb}{0.2,0.4,0.0}
+\providecolor{dgraycolor}{rgb}{0.30,0.3,0.30}
 %    \end{macrocode}
 % Unsurprisingly, the |\sageplot| command works poorly without graphics
 % support.

[dimpase]

Changed branch from u/jhpalmieri/packages/sagetex/v3.6 to u/dimpase/packages/sagetex/v3.6

[dimpase]

Changed commit from 48f33a2 to b6ba36a

[dimpase]

comment:36

this is the fixed v3.6.

One still has to deal with comment:31

---

New commits:

b6ba36a

fixed sagetex v3.6

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from b6ba36a to 2cd5b89

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

df57cff	update sagetex to version 3.6
005e443	correct version in setup.py, and deps
ab7aea4	trac 32887: change location of sagetex to venv/share/texmf/...
2cd5b89	fixed sagetex v3.6

[dimpase]

comment:38

rebased over latest beta

[mkoeppe]

comment:39

That build/pkgs/sagetex/install-requires.txt exists (which triggers installation in SAGE_VENV instead of SAGE_LOCAL) is a mistake introduced in #30719. sagetex does not exist on PyPI, so the contents of install-requires.txt are wrong.

Overall, I'm skeptical about treating sagetex as a Python package. As the instructions in https://doc.sagemath.org/html/en/tutorial/sagetex.html#using-sagetex show, after installing it with pip (or Sage package), still manual installation work is needed for the TeX style files. So I would suggest to refocus sagetex as a TeX package and consider CTAN and texlive as the main distribution strategy. Python is merely used as a scripting language.

The section https://doc.sagemath.org/html/en/tutorial/sagetex.html#using-sagetex should just be moved to the sagetex README.

I would think that sagetex works with any version of Sage released in the past 5 years at least. This compatibility should probably be documented in sagetex.

(Disclaimer: I have never used sagetex and don't plan to use it.)

[jhpalmieri]

comment:40

If it gets installed in SAGE_LOCAL, then my changes to the documentation need to be reverted.

[jhpalmieri]

comment:41

The new version doesn't build for me:

[sagetex-3.6]   error: can't copy 'sagetex.pdf': doesn't exist or not a regular file

and indeed that file is missing from the tarball.