loadDom SIGSEGV: Illegal storage access.

[stellarskylark]

Hello, I'm getting an error trying to load a dom from a filestream:

~/software/rpgsheet (master*) » nimble build
  Verifying dependencies for rpgsheet@0.0.1
      Info: Dependency on yaml@>= 0.16.0 already satisfied
  Verifying dependencies for yaml@0.16.0
   Building rpgsheet/rpgsheet using c backend
/home/skylar/software/rpgsheet/src/rpgsheet.nim(56, 37) template/generic instantiation of `loadDom` from here
/home/skylar/.nimble/pkgs/yaml-0.16.0/yaml/dom.nim(195, 3) Warning: Cannot prove that 'result' is initialized. This will become a compile time error in the future. [ProveInit]
-------------------------------------------------------------------
~/software/rpgsheet (master*) » ./rpgsheet examplesheet.yaml DEX
[NimYAML] Error in file dom.nim at line 188:
assertion failed!
[NimYAML] Stacktrace:
Traceback (most recent call last)
/home/skylar/software/rpgsheet/src/rpgsheet.nim(56) rpgsheet
/home/skylar/.nimble/pkgs/yaml-0.16.0/yaml/dom.nim(202) loadDom
/home/skylar/.nimble/pkgs/yaml-0.16.0/yaml/private/internal.nim(37) compose
Traceback (most recent call last)
/home/skylar/software/rpgsheet/src/rpgsheet.nim(56) rpgsheet
/home/skylar/.nimble/pkgs/yaml-0.16.0/yaml/dom.nim(202) loadDom
/home/skylar/.nimble/pkgs/yaml-0.16.0/yaml/private/internal.nim(39) compose
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
[1]    55450 segmentation fault (core dumped)  ./rpgsheet examplesheet.yaml DEX

And here are the lines of code that trigger the error:

var file = openFileStream(paramStr(1))
var sheetDoc: YamlDocument = loadDom(file)

An earlier version of my solution used loadToJson with no errors at all, so I think it's specifically a bug of some kind in the loadDom function. It is also strange that I'm getting the "template/generic instantiation of loadDom" error when there's only one definition of it documented, and it clearly picks the right one based on the stacktrace.

I'm using Nim 1.6.6.

[stellarskylark]

Oh god never mind somehow examplesheet.yaml became an empty file, whoops.

[flyx]

Well even if examplesheet.yaml did become an empty file, a failed assertion is not how NimYAML should behave in that case. And empty file is still a valid YAML document that shall be loadable via loadDom. I'll reopen the issue to address that error.

btw you get the instantiation message because loadDom takes a variable typed s: Stream | string which basically means there are two instantiations of loadDom, one for each of the given types.

Also nice to see someone is using NimYAML for an RPG sheet, that was the initial reason I wrote it though my plans got altered a lot and the project went in another direction. :)

[stellarskylark]

Also nice to see someone is using NimYAML for an RPG sheet, that was the initial reason I wrote it though my plans got altered a lot and the project went in another direction. :)

Oh cool! Yeah I've been working on a general-purpose RPG character sheet application for the terminal. Here's the repo if you're interested! https://git.skylarhill.me/skylar/rpgsheet

[flyx]

Concerning this issue, I will shortly push a fix that is basically an API change that deprecates loadDom and dumpDom, instead you can just use load and dump with the YamlNode type. This has several advantages, including that you can deserialize parts of your YAML into YamlNode while other parts can be loaded into native Nim objects. The old API stays available for now, but will have deprecation warnings on it.

---

Here's the story of my pursuit for something similar to what you do so far, if you're interested (feel free to ignore this or come to different conclusions, I certainly don't want to tell you something is a bad idea):

I wrote NimYAML with the goal of writing a GUI-based character generator, sheet and manager for The Dark Eye, which is what we play in Germany. After NimYAML was finished, I expanded the scope: I wanted to have a library, named libheroes or something, that understands declarative descriptions of RPG character sheets (similar to what you're doing). It would be able to read in instances of those declaration, i.e. you'd have a YAML file describing how a DnD character sheet would look like, and then you'd have a YAML file describing the stats of your character, using the structure defined in the general character sheet description.

The API of that library would be so that you could implement any UI on top of it, being textual or graphical. This should in particular enable existing tools like e.g. Roll20 to consume such character sheets, if they want to. At this point, I decided not to use Nim because no-one would adopt a library with a Nim API, and writing a Nim library with a C API seemed to be a horrible idea. So I wrote it in C.

The result was actually usable, had a complete evaluation stack for calculating derived values and such. However I realized that YAML was, generally, a horrible input language for doing all this. A lot of people are not even able to write simple YAML correctly, and a RPG character sheet is usually quite complex in its mechanisms (particularly if you want to define how to generate a character, so that the lib can support you with that).

At that point, I also happened to be frustrated with my job and so decided to try doing a PhD thesis on designing a better data input language for such things. I plan to release that in 1-2 months.

I never released all that C code, but it's still lying around, so if you think it could help you with your project, I can give you access. I don't really want to make it public since I will not continue that work and don't want people to jump onto a dead project (I will redo everything with my new language).

[stellarskylark]

That's really cool! I think for me it was basically a choice between JSON or YAML, and as far as human-readability the choice is pretty obvious there. I agree that YAML doesn't feel super ideal though; I'd be very interested in reading your paper when it's ready!

[flyx]

Thing is, I won't finish my PhD thesis. I wanted to do usability studies and all that stuff with students, and that never happened partly because of COVID. So all I will release is the spec and reference implementation of my language. I'll notify you when that happens.

Here's how an in-progress definition of a sheet looked with my C lib. You can see how I had a section for declaring data types, and I allowed to declare multiple sheets per system (so you can e.g. have a separate sheet for a familiar with a different structure). Also you can see that I used a second YAML document in the stream to define user-facing names for all the stuff, which was designed to evolve into i18n support.

[flyx]

@stellarskylark Since you expressed interest in the language I design, here's a link. It has been primarily been designed to substitute LaTeX, which has a somewhat different use-case, but it does allow you to input general structured data and so is generally suitable. In particular, since functions are first-class values, you can easily input formulas for derived values on a character sheet.

In this early stage, its prime functionality is text in, text out – though you can already get a parsed document via its API. A C API that can be called from languages other than Zig is planned, so are a lot of other features. There might be a proper release before the end of the year; currently it's more of a showcase.