Loading Package information from CTAN

sunderme commented 1 year ago

CTAN is a huge latex packaage database. Apart from providing the actual packages, it also contains a caption and a short description on every package.

The idea is to download that data to provide info for all packages within txs.

Describe the feature and the current behavior/state

Load caption/description from CTAN to provide package information for quickstart wizard, package insertion via menu or with the completer.

Who will benefit with this feature?

Beginners who need some more introductory information on some packages.

Any Other info

sunderme commented 1 year ago

CTAN offers a rest json api. e.g.: https://www.ctan.org/json/2.0/pkg/mathtools

sunderme commented 1 year ago

they also offer a categorized overview which may be interesting for a kind of package wizard ?

https://www.ctan.org/topic/table-long

muzimuzhi commented 1 year ago

CTAN offers a rest json api. e.g.: https://www.ctan.org/json/2.0/pkg/mathtools

see also https://www.ctan.org/help/json/2.0/pkg

Corner cases: The CTAN package name may not be the same as the name in .sty or .cls files.

There're CTAN bundle packages that provide several .sty and/or .cls. For example, CTAN package amsmath provides at least amsmath.sty and amscd.sty, which are designed to load separately. And of cause the koma-script, tikz, and zref CTAN packages.
There're CTAN packages that each of them does only provide a single .sty or .cls file, but with a file name different from the corresponding CTAN package name. I can't name one of them right now, but if my remember is alright...

To sum up, a mapping from .sty/.cls (base) file names to CTAN package names is needed. Also note that CTAN's current searching API doesn't support file search, see https://www.ctan.org/help/json/2.0/searching .

mbertucci47 commented 1 year ago

There're CTAN packages that each of them does only provide a single .sty or .cls file, but with a file name different from the corresponding CTAN package name. I can't name one of them right now, but if my remember is alright...

For example erewhon-math provides fourier-otf.sty, xcharter-math provides xcharter-otf.sty.

octaeder commented 1 year ago

What about the Packages Help in the Help menu? A quite interesting feature.

muzimuzhi commented 1 year ago

There're CTAN packages that each of them does only provide a single .sty or .cls file, but with a file name different from the corresponding CTAN package name. I can't name one of them right now, but if my remember is alright...

For example erewhon-math provides fourier-otf.sty, xcharter-math provides xcharter-otf.sty.

Another example: CTAN package newtx (https://ctan.org/pkg/newtx) provides newtx.sty, newtxtext.sty, and newtxmath.sty.

octaeder commented 1 year ago

This shows the description:

grafik

and you can open the document. Do we need more?

octaeder commented 1 year ago

When I understand correctly this case is about presenting information about packages. I see the following:

Currently txs uses a packageList[^1] that is loaded from github. I don't know how this packageList is updated. It is used to present captions (short descriptions) for packages in the texdoc dialog[^2]. As examples, I can't find packages array and bookmark in the packageList, others are missing a description[^3]. Maybe a ctan list with packages and captions should be used[^4].

The texdoc dialog doesn't present the full list. It receives a string list when it is called[^5]. Packages known to the completer having completion words are added to the string list. This seems to be some heuristics for detecting not existing packages. Thus package datetime doesn't find the way into the packageList[^6]. Probably this heuristics can be removed when the ctan list is used.

Package caption is presented in tooltips in the text, again not for bookmark, but for datetime (no heuristics used here). Txs helps open the package documentation. Interestingly the texdoc dialog opens it with an external pdf viewer (acrobat), while with the context menu it is opened with the internal viewer. The later has the disadvantage that it replaces the document you are currently viewing.

I made a quick and dirty change of the current dialog into a table that may give an impression how it could be. Links directly to the ctan package description could be added:

grafik

[^1]: s. latexrepository.cpp [^2]: Menu Help/Packages Help... [^3]: examples: adjustbox, ccicons have no caption, c90 is in the list but I can't find it on ctan [^4]: s. https://www.ctan.org/xml/2.0/packages [^5]: s. texstudio.cpp [^6]: you may select (mark) package datetime (or any other) in the text, then the started dialog will show the package anyway.

octaeder commented 1 year ago

grafik

Pressing the left button will open url https://www.ctan.org/pkg/eurosym.

sunderme commented 1 year ago

not sure, that I want an extra button for that. After all the dialog is "texdoc help" ...

octaeder commented 1 year ago

and the rest is ok?

sunderme commented 1 year ago

That the caption is filled ? Yes.

sunderme commented 1 year ago

so maybe using documentation from ctan if not installed locally ?

sunderme commented 1 year ago

but a filter/search field is missing. You don't want to scroll through hundreds of packages.

octaeder commented 1 year ago

the package table is sorted and contains the intersection of the packages found by the completer and the ctan package list. So in principle it's not difficult to scroll to the location. And when you mark the package in the text and then open the dialog then the list is located at the package. Can you give the largest number of packages you have seen in documents? Adding the field of course would be no problem.

CTAN is reasonable, but then I see no reason for a cut down to not installed packages.

Remarks for the CTAN packages.xml: I wrote a simple parser because the example shown suggests that the file can be parsed easily. But then I found that some of the package tags have one line break:

  <package key="simple-thesis-dissertation" name="simple-thesis-dissertation" caption="Template for a simple thesis or dissertation (Ph.D. or master&apos;s degree) or technical
  report, in XeLaTeX" />

I solved this by editing the file to remove the line breaks. The file is stored in utilities folder of the repo. texstudio.exe reads it from the folder where the exe is located. So there is no need for internet connection (same argument u used for manual, latex reference ...).

sunderme commented 1 year ago

qt offers xml parser which is used several times in txs. No need to invent the wheel again, especially as it covers corner cases in the xml encoding.

sunderme commented 1 year ago

even my small test document has already 20 packages (with included packages), that can grow quite quickly for more complex packages like tikz etc.

sunderme commented 1 year ago

by the way, search is actually the point where things start to get interesting. If you have descriptions for all packages on CTAN, you can search for table packages, etc directly.

octaeder commented 1 year ago

Now a QXMLreader is used to read packages.xml. I added a search field where you can enter a (regExp for a) name to search (s. toolTip):

For the moment I would like to come to an end with this change that clearly improves the old dialog (some statistics for my test case):

number of packages listed: 88 (old) vs. 87 (new)
17 old packages not found at CTAN, especially some tikz... packages, where now pgf is in the list

sunderme commented 1 year ago

sure

octaeder commented 1 year ago

The file is stored in utilities folder of the repo. texstudio.exe reads it from the folder where the exe is located. So there is no need for internet connection (same argument u used for manual, latex reference ...).

Just to be sure: You are aware of this? This means that you have to change things such that the file is available locally after installing txs, in portable version ...

texstudio-org / texstudio