search

KhronosGroup / OpenGL-Registry

OpenGL, OpenGL ES, and OpenGL ES-SC API and Extension Registry
678 stars 274 forks source link

Source text or machine-readable data for state tables #571

Closed hikari-no-yume closed 1 year ago

hikari-no-yume commented 1 year ago

The state tables section of the OpenGL specification is a very handy reference, but so far as I can tell it's only available in PDF form. Is there no machine-readable (e.g. XML) form somewhere, or even something like LaTeX source? It would be very useful when writing tooling. In particular, I'm interested in doing API translation, and with a machine-readable form I could have a script automatically compile common subsets, for example. (I asked about this on a social media site and someone suggested making this issue.)

oddhack commented 1 year ago

The table only exists as LaTeX markup. The WG might be OK with releasing just the state table appendix LaTeX, which is a question for @pdaniell-nv . If we were to do this you should not expect much help interpreting it, nor for it to be buildable as a standalone document, because we're not putting many cycles into GL these days.

Alternatively, you can 'pdfgrep .' on relevant pages of the PDF to extract the text followed by some hand-massaging of the result. I would not recommend trying to cut/paste from a PDF reader as (at least with evince) the cell order seems to get scrambled.

hikari-no-yume commented 1 year ago

The WG might be OK with releasing just the state table appendix LaTeX

If so, I'd really appreciate it!

If we were to do this you should not expect much help interpreting it, nor for it to be buildable as a standalone document, because we're not putting many cycles into GL these days.

That's no problem for me. I'm sure I could figure it out on my own.

Alternatively, you can 'pdfgrep .' on relevant pages of the PDF to extract the text followed by some hand-massaging of the result.

I definitely could attempt this, but my fear is it would be a very fragile solution at best.

hikari-no-yume commented 1 year ago

Oh, in case anything does come of this: is there distinct source for each OpenGL version, and do you think it would be more effort to provide the state tables for all versions, not just the latest version?

oddhack commented 1 year ago

Oh, in case anything does come of this: is there distinct source for each OpenGL version, and do you think it would be more effort to provide the state tables for all versions, not just the latest version?

They are separate branches in our internal development tree though it's increasingly rare that we would update anything but the current 4.6 spec (and it's unlikely there will ever be a (edit: new) core version in the future).

I don't think there's a lot of value in the older tables and would prefer not to have to go dig them all up if the WG does decide to proceed with this. gl.xml describes which state variables are introduced with each version of the API although there are occasionally updates in the limits between revisions of the API.

oddhack commented 1 year ago

Also, looking at your touchHLE issue, it sounds like you may want the GLES state tables, instead of or in addition to the GL ones?

hikari-no-yume commented 1 year ago

They are separate branches in our internal development tree though it's increasingly rare that we would update anything but the current 4.6 spec (and it's unlikely there will ever be a (edit: new) core version in the future).

I see, thanks for the insight.

Also, looking at your touchHLE issue, it sounds like you may want the GLES state tables, instead of or in addition to the GL ones?

Yes, that's right. For my current application, I'm interested in the intersection of OpenGL ES 1.1 and the OpenGL compatibility profile (notionally OpenGL 2.1). Since OpenGL ES isn't a strict subset of OpenGL, it would be useful to have state tables for both.

I don't think there's a lot of value in the older tables and would prefer not to have to go dig them all up if the WG does decide to proceed with this. gl.xml describes which state variables are introduced with each version of the API although there are occasionally updates in the limits between revisions of the API.

I think the gl.xml approach means you could skip most of the old versions, yeah. At least for my application, I don't think the limit changes matter, it's really the parameter names and type information I'm after.

The only caveat I can see is that major versions sometimes remove old features:

I don't have an immediate use for them, but for completeness, I'm sure the GL 4.6 core profile (if it's not a strict subset?) and GLES 3.2 would be useful to others.

Someone tell me if I'm forgetting some compatibility break.

pdaniell-nv commented 1 year ago

We discussed this in the OpenGL/ES working group and we don't object to making the table source public. I'll leave it to @oddhack to figure out the details on how best to do this to meet your needs and not be too much work.

hikari-no-yume commented 1 year ago

Thanks for taking the time to consider it!

oddhack commented 1 year ago

I'm attaching the statetables.zip LaTeX markup for the current GL 4.6, ES 3.2, and ES 1.1 state table appendices. They are not standalone documents as they use some formatting macros whose definitions are not included and which I really can't justify the time to extract from the rest of the spec build, but they are fairly obvious. N.b. the '\ifnum\specdep=1' macros surround material that only goes in the compatibility specification build.

hikari-no-yume commented 1 year ago

Thank you! To confirm, are these for OpenGL 4.6 and OpenGL ES 3.2, respectively? Would you be able to provide OpenGL ES 1.1 as well?

oddhack commented 1 year ago

Thank you! To confirm, are these for OpenGL 4.6 and OpenGL ES 3.2, respectively? Would you be able to provide OpenGL ES 1.1 as well?

Yes. I updated the zipfile attachment above to include that. I cannot provide the tables for all older versions of the specifications, however.

hikari-no-yume commented 1 year ago

Thank you! Those three are all I needed, so I think you can consider the request fulfilled now. I particularly appreciate that there's a license header on each of the files, so it's clear what I can do with them.

I noticed the OpenGL ES 1.1 LaTeX doesn't include the introduction text (“The tables on the following pages indicate …”) and the table explaining the type codes. Was that intended? It's not a big deal since I can fill in the gaps using the other two files.

In any case, I've now written a proof-of-concept which pattern-matches the LaTeX and produces (very ugly) HTML from it. So the state tables LaTeX is definitely enough for what I want to do.

My script is quite hacky and needs some cleanup, e.g. deprecated variables are ignored right now. I intend to return to it later and make it produce nice JSON or CSV or something, and when I've done that I'll publish it properly and leave a link here for others' reference. In case I never get around to it, though, please consider the PoC to be MIT licensed (SPDX definition).

oddhack commented 1 year ago

I noticed the OpenGL ES 1.1 LaTeX doesn't include the introduction text (“The tables on the following pages indicate …”) and the table explaining the type codes. Was that intended?

In the sense that the ES 1.1 spec is many years older than the other two and periodic reformatting / restructuring of the document happened, yes. Glad this is working out for you.

hikari-no-yume commented 1 year ago

Here's what came of my efforts:

As expected, it was far from trivial to parse, but with a few days' effort I got some of the fields into a good shape. It's still a work in progress and can't parse everything, but it parses enough that it felt worth releasing.

This will be my last comment in this ticket. Thanks again for releasing the sources!