search

erkyrath / remglk

A remote-procedure-call implementation of the Glk IF API
http://eblong.com/zarf/glk/index.html
MIT License
26 stars 14 forks source link

Graphics/sound support #6

Open erkyrath opened 7 years ago

erkyrath commented 7 years ago

The JSON graphics commands have already been specced out and implemented in GlkOte. We would need analogous sound commands too.

Gestalt handling is a little weird. In principle RemGlk cannot know whether its display partner supports that stuff. (Sleepmask/Floyd is an obvious case that cannot!) So I think we'll want command-line options for RemGlk to say "graphics, sound, timer support: yes/no". That will determine the gestalt responses, and also whether RemGlk bothers to kick out those JSON commands.

BroadcastGames commented 7 years ago

Perhaps HyperLinks too. Unless I compiled it wrong or something, right now HyperLinks crash remote-if.py if the game has them. For example, CityofSecrets_v3 on Git interpreter if you type the 'about' command the webserver crashes.

EDIT: I see that running with RemGlk param "-stderr" keeps it from crashing on encountering a HyperLink.

With "-stderr" on, the CityOfSecrets v3 menu on the bottom actually work pretty well on GlkOte. You have to fiddle the cursor to a blank spot. For example, start the story and do "z", "z", "z" and a Dark Lady will ask you a question which you can pick prompts. I was able to get my cursor into that window and use the arrow up/down and number key. Anyway, figured that positive experience was worth sharing.

erkyrath commented 7 years ago

Formally, this is a bug in CityofSecrets_v3. It should not make hyperlink requests on an interpreter that does not support them.

However, you're right that hyperlink support should be added. That's the easy one.

erkyrath commented 7 years ago

Timer support is in, and works correctly with glkote amd regtest.py.

curiousdannii commented 7 years ago

Could GlkOte send a list of what it supports in its 'init' message? I'm looking at using glkapi.js in node and I think the only think it has which won't work in node is the has_canvas check. But if remglk could also get its information from glkote then that would be simpler than having to specify several command line arguments. (Though they could be kept to overwrite it I guess.)

GlkOte should probably also send its version number in its init message so that remote systems can check it.

erkyrath commented 7 years ago

Yes, the support list is a good idea.

Not sure about the version number. I generally like version transparency, but RemGlk can't do anything with that information.

erkyrath commented 7 years ago

...now that I think about it, it might not even be GlkOte on the other end of the line. (Regtest.py and Sleepmask use the same protocol to talk to RemGlk.) So relying on a version number for anything is dangerous.

curiousdannii commented 7 years ago

Maybe a version number for the protocol then? I was thinking of situations like the timer function where a VM might want to hand it off to the remote IO implementation but be unable to tell whether that is supported. Of course now we can tell because of the support array, but something else like that might arise again. Or when such a problem arises we can deal with it then.

erkyrath commented 7 years ago

The support array seems like the right way to handle this. It covers RemGlk's needs, anyhow.

erkyrath commented 7 years ago

Hyperlink support implemented.

BroadcastGames commented 7 years ago

On graphics and sound, is there anticipation that RemGlk will be responsible to extract graphic and sound payload elements from the blorb file to hand off to middleware? Write binary files to disk in a data directory and send paths - or putting base64 of the elements into the JSON stream?

erkyrath commented 7 years ago

No, I think that image/sound extracting is better handled by the application framework or launcher.

RemGlk will refer to images only by Blorb number. (See http://eblong.com/zarf/glk/glkote/docs.html#graphicsupdate , and also farther down under "Line Data".) Before launch, the display code will want to unpack all the resources. https://github.com/erkyrath/glk-dev/blob/master/blorbtool.py has a sample implementation of this.

BroadcastGames commented 7 years ago

Ok, Thank you for explaining your direction on the asset management.

For testing sake, i think this story has the kind of complex windowing and HyperLink support in a complex-but-understandable way: http://ifdb.tads.org/viewgame?id=q5el4dphbf7q6e5y GlkOte almost runs it (perfectly) currently but I think it must violate the issue of window Input origin when you click the HyperLink because that generates an error. That story is almost 10 years old, I'm trying to scratch together working Inform 7 source code to reproduce that exact layout.

erkyrath commented 7 years ago

There are, unfortunately, a lot of games that make invalid calls which are not reported in the IDE interpreter or Gargoyle. I recommend testing in Lectrote.

curiousdannii commented 7 years ago

Could mouse be added to the support array? I'm not sure if any of the other gestalts would be helped by also being included, but mouse should I think.

Also, you have a typo in glkapi: the gestalt for gestalt_MouseInput checks wintype_TextBuffer instead of wintype_TextGrid.

erkyrath commented 7 years ago

I am adding these values as they go into remglk, since remglk is the only client for them (as far as I know).

curiousdannii commented 7 years ago

I've just published https://github.com/curiousdannii/glkote-term :)

GlkOte-term will send different support arrays to GlkOte, and I intend on consuming them in ZVM. It could be used by Quixe too perhaps, or my eventual GVM.

[Edit] I'm looking at what needs to be changed for glkapi.js to be completely node friendly, and I've noticed that its gestalt support for graphics is interesting. If win is a buffer window then it returns true - if it's a graphics window then it checks for canvas support. Should we consider a glkote support array graphics entry to mean full/holistic graphics support?

BroadcastGames commented 7 years ago

Revisiting my comment from December 13: (Current verisons of) GarGlk and Quixe both have trouble with the bottom hints window Hyperlinks in Dead Cities - example: http://iplayif.com/?story=http%3A%2F%2Fwww.ifarchive.org%2Fif-archive%2Fgames%2Fglulx%2FDeadCities.gblorb Jon Ingold spent a lot of time open sourcing his Inform extensions and writing about them. Anyway, I guess 10 years ago when he ran this story - it did indeed work on the available Glulx interpreters...

RemGlk is very concerned that the hyperlink come in with the right window and generation. Any thoughts on how to make this story work and perhaps detect others with the same problem?

Anyone have other stories to recommend for Hyperlink testing with multiple windows?

BroadcastGames commented 7 years ago

Graphics support in RemGlk is working great. The Moon Watch story proved to be one of the most challenging beyond Dark Cities and it's 20+ Glk windows. If you want to see RemGlk in action on Android - I've uploaded progress so far, join the Alpha Experimental of my code:

https://play.google.com/apps/testing/com.wakereality.thunderword.experimental

WARNING: My app works, but needs a lot more refinement. Right now it's best to use a tablet and a USB/Bluetooth keyboard. I have a lot more usability labor to do to get the on-screen touch keyboard and adaptable layouts to something approaching sensible.

If you don't have an Android device, the Amazon Fire (generation 5) tablet I am working with is decent enough (Android 5.1) and under $50 retail. It can't, on it's own, use the Google Play store, but I can give you (upload here) the APK for side-loading (easy). Or you can modify the tablet to work with Google Play store with some hacking.

Thank you for all the work and the long-tradition of cross-platform Glk and bring interpreters together!