Support for non-Blorb resources (patches included)

[GoogleCodeExporter]

Currently garglk can load resources from games packaged in the Blorb format. 
Not all games have their resources packaged in that format though. As a result, 
games made with ADRIFT (SCARE engine) or Quest (Geas engine) cannot display 
images in garglk.

Here is a set of patches that adds image support for Geas and SCARE. The first 
patch adds generic support for loading from plain file resources. The second 
patch adds support in Geas for parsing the "picture" command and to make the 
(gar)glk calls to display the image. The third patch adds support in SCARE to 
display images using (gar)glk.

The generic patch keeps all registered resources in a simple linked list. This 
is not terribly efficient in terms of lookups, but unless a game uses many 
thousands of images performance should not become an issue. The generic code 
keeps track of the file path, the start offset and the data length. For images 
the data length is not used, but it might be required for other types of 
resources. If it is never required, the code can be simplified a bit: the 
stat() call in resource_from_file() would not be needed then.

The Geas patch does not support displaying images at resolutions other than 
their native resolution. I wouldn't know why a game would include images at a 
different resolution than they will be shown at, but the Quest format seems to 
support it. If required I could add support for this, but it didn't seem 
important enough to me to spend time on.

The SCARE patch renames "LINUX_GRAPHICS" to "XV_GRAPHICS", since xv is not 
Linux-specific, nor is it likely that modern Linux distributions will install 
xv by default. Maybe the option of spawning an external image viewer can be 
removed altogether? Inline images provide a far superior user experience, in my 
opinion.

By the way, the Hugo engine works around lack of plain file resource support by 
building a blorb file in a temporary directory and then loading from that. This 
seems like a less efficient approach to me, so perhaps the Hugo code could be 
modified to use the new plain file resource loading as well.

Please review the patches. They work fine on my Linux system with GTK, but I 
don't know if I took sufficient precautions to not break different systems. For 
example, is stat() portable enough? Is it safe to call Glk image functions on 
text-only systems?

Original issue reported on code.google.com by mthuu...@gmail.com on 10 Feb 2012 at 12:54

Attachments:

• fileresource.diff
• geas-images.diff
• scare-images.diff

[GoogleCodeExporter]

Another thing to look at while reviewing: should glk_window_flow_break() be 
called before, after, or before and after glk_image_draw()?

Original comment by mthuu...@gmail.com on 10 Feb 2012 at 1:00

[GoogleCodeExporter]

It seems ADRIFT can use either images inside the game file or as separate 
files. I'll update the SCARE patch to support this.

Original comment by mthuu...@gmail.com on 10 Feb 2012 at 4:49

[GoogleCodeExporter]

Here is an updated version of the SCARE patch. It supports images in separate 
files now.

Original comment by mthuu...@gmail.com on 11 Feb 2012 at 12:57

Attachments:

• scare-images2.diff

[GoogleCodeExporter]

Hi Marteen,

I apologize for my delayed response; the Google Code mechanism for notifying me 
about new issues seems to be broken.

I see the point of adding a richer set of file handling functions to garglk but 
I would prefer not to incorporate those features at this time. Interpreters 
which depend on that functionality would require Gargoyle instead of remaining 
neutral with respect to the Glk implementation.

Although this project hosts and serves as the de-facto repository for many 
interpreters, one of my goals is to avoid making those interpreters depend on 
Gargoyle except in cases where the features cannot otherwise be implemented.

I respect the work you've put into this and especially your willingness to work 
on SCARE and Geas - they can certainly use the love. If you rework your changes 
to touch only the interpreter code, I will gladly accept the patches. 

Regards,
Ben

Original comment by bcressey@gmail.com on 13 Jun 2012 at 6:25

[GoogleCodeExporter]

I'm willing to rework the patches. However, I currently don't see how this 
could be implemented without adding any new API elements at all between the 
interpreter and the rest of the program.

When an interpreter calls functions such as glk_image_draw(), it passes a 
resource number. The consumption of the bytes identified by that resource 
number is done by the Glk implementation, but the knowledge needed to fetch 
those bytes is interpreter specific. So there must be some way for the 
interpreter to register a resource; that is what the proposed 
resource_from_file() and resource_from_file_segment() do.

Blorb is special because it is an interpreter-independent resource format. 
Therefore, Blorb resources do not have to be registered by the interpreter, so 
no resource registration API is needed. We also want to run game files that do 
not use Blorb though, so "use Blorb" is not a satisfying solution.

The Hugo interpreter tries to work around this issue by packaging its native 
resources into Blorb format at runtime. Creating such a temporary Blorb file is 
an inefficient approach though: it is potentially a lot of data that has to be 
copied over and a binary index and metadata representation is generated only to 
be parsed again later. Also, creating the temporary file requires use of a 
platform specific I/O API, while the whole point of using Glk I/O is to have 
platform independent I/O, so this approach defeats the purpose, in my opinion.

Maybe I overlooked a possibility to implement non-Blorb resources without 
adding anything to the API. If so, please enlighten me. Otherwise, I guess we 
should be discussing what kind of API additions would create the smallest 
burden for someone porting an interpreter to a different Glk implementation.

Original comment by mthuu...@gmail.com on 14 Jun 2012 at 12:45

[GoogleCodeExporter]

You're right; I overlooked the resource number twist. Let me review the patches 
with that in mind.

Original comment by bcressey@gmail.com on 14 Jun 2012 at 6:07

• Changed state: Accepted

[GoogleCodeExporter]

OK, I had a mental block because I knew that the Level 9 interpreter drew 
graphics without any special Glk extensions. So I took a closer look - 
essentially it uses an empty Glk window as a canvas, then draws the bitmap 
pixel by pixel. Ugh.

I'd like your thoughts on an alternate approach: providing API functions for 
adding resources to an in-memory blorb representation that Gargoyle would build 
on behalf of the interpreter:

garblorb_initialize_resource_map()
garblorb_add_resource()
garblorb_finalize_resource_map()

Where add_resource() would have semantics similar to your proposed 
resource_from_file_segment function.

The disadvantage is that all the game's resources would be resident in memory, 
but this is true today for any game which packs the story file into the blorb.

On the Gargoyle side this requires some changes in both imgload.c and sndsdl.c. 
We could assume at the resource loading stage that the resource would always be 
in blorb form. The loader code would need to distinguish between the in-memory 
and on-disk cases unless we abandon the cgblorb.c implementation and hoist the 
blorb into memory in all cases.

I'm primarily concerned with the breakage that might result if the behavior of 
finding non-blorb images and sounds as PIC# and SND# in the current working 
directory is altered. The main consumers of this code path are unpacked blorb 
games during a development / debug cycle.

The plan would be to use those APIs internally on startup, if Gargoyle found 
itself running without a blorb in a working directory with PIC* or SND* files.

I won't make you write the Gargoyle side of this but I would like your 
feedback. The main disadvantage is that this would only really support resource 
initialization at one time; game formats with no predefined resource list that 
rely on the ability to generate ad-hoc resources at runtime would be 
unsupported. 

Original comment by bcressey@gmail.com on 14 Jun 2012 at 8:11

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Issue 138 has been merged into this issue.

Original comment by bcressey@gmail.com on 14 Jun 2012 at 8:31

[GoogleCodeExporter]

Not all formats have a resource map in them. From a quick look at the 
interpreter code, it seems ADRIFT does have a resource map, but Quest does not.

It might be possible to build one on startup by analyzing all the game code or 
by scanning the game directory for potential resources, but that is a lot more 
work than registering a resource on demand. Also the code analysis approach 
would break if there are games using computed resource names, but I don't know 
if there are interpreters capable of that and if so, games that actually use it.

Is it essential to finalize the resource map before resources can be used?

One way to avoid breaking the PIC#/SND# system is to make the interpreter glue 
explicitly select which resource system to use: Blorb or plain file segments. 
If Blorb is selected, resources are loaded from either an actual Blorb file or 
an unpacked Blorb-like directory of PIC#/SND# files. If plain file segments are 
selected, resources are loaded from the provided file names and resource 
numbers are only used to identify resources internally.
(My original patch does break the PIC#/SND# system.)

Whether the resources are kept in memory or on disk is not determined by the 
API, if the API accepts filename + offset + length. It is an implementation 
choice whether to load the data immediately or store filename + offset + length 
and load it later.

Original comment by mthuu...@gmail.com on 14 Jun 2012 at 11:40