C64 formats load/save

[GoogleCodeExporter]

To start with, this looks like it's THE program I've been looking for.
Together with the c64 "restrictions mode" in another issue it sure is the
killer app.

But not without load/save.

I've made some conversion programs for c64 graphics, and after looking at
the documentation I could make some load/save routines in "no time", but
probably just not today.

Standard hires and multicolor is easy to start with.
I also made a converter that saves as "text plus font", was quite interesting.

Now the questions are how we want all this to be saved. 
Some times at least I want pixel data separated from colour data in
different files, and a 2 color image should probably ignore color data.
(This has disturbed me in gimp, when it remaps the colors in some screen
blocks, making the picture useless for some later effects)
Here it COULD be interesting to even load pixel/color data independent of
each other.

Some comments (and acceptance) is all I need to start coding and submitting
patches when I have something nice, clean and useful.

Original issue reported on code.google.com by mumu21...@gmail.com on 1 Sep 2009 at 4:48

[GoogleCodeExporter]

Ok, stubs are built.
There are quite some restrictions for the c64 formats, but I've found no way to 
tell
the user _what_ error occurred. Is stderr/stdout ok for this?

Original comment by mumu21...@gmail.com on 1 Sep 2009 at 6:47

[GoogleCodeExporter]

Use Warning_message("something bad hapenned") to display a message box. Use 
Verbose_error_message if your message is so long that it needs automatic word 
wrapping. These are defined in windows.h.
Usually we don't use stderr/stdout anywhere except in DEBUG() and Error().

About pixel vs color data, there is support for this, you just set dont_save or 
something like that to true to say that you saved only the picture data or only 
the 
pixels (this is documented in the wiki page about howto write a save or load 
function). If you save both, but in two different files, I see no problem. You 
can 
make the user save the file to some .scr file, and save the pixel data as 
"file.pal" 
or "file.scr.pal". Now, if the pal-file already exist, you may overwrite it and 
we 
won't check for it automatically. Alternatives are :
-Make the user-selected filename be actually a folder or a zipfile, and put 
everything inside it. Folders are not available in standard C and zipfiles need 
zlib, so i'd prefer the zip solution (we already need zlib for png).
-Save only the pixeldata in one format, only the palette in another. Make the 
user 
need to save things twice. But he can work in png and convert the files only 
when he 
need to see the picture on the real hardware.

In any case, I'd say, go for it, we'll apply the patches.

Oh and, if possible, i'd like to avoid extensions such as .scr. Just because 
this is 
used as the default name on amstrad CPC, so ther will be conflicts. Maybe using 
.c64 
instead would be fine. But I don't know what are the usual extentions on c64 :)

Original comment by pulkoma...@gmail.com on 1 Sep 2009 at 7:48

• Changed state: Accepted

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Progress update: I'm saving and loading 16 color hires pictures now.

File extension is "c64" right now, perhaps we can make this some kind of "?64" 
to
distinguish between different c64 formats.

Save-format is determined from resolution (320 or 160 pixels), and during load 
I have
to go for file size I guess, as the multicolor pictures needs some extra color 
data.
Or filename?

However, I'm still open for ideas which might be better than mine.

Original comment by mumu21...@gmail.com on 4 Sep 2009 at 11:00

[GoogleCodeExporter]

almost done with multicolor!
it's just the backgroundcolor I have to save too.
and some code clean up.

What diff format do you want? "svn diff" or some other flags?

Original comment by mumu21...@gmail.com on 4 Sep 2009 at 12:58

[GoogleCodeExporter]

How do I update the Pixel_ratio without manually resizing the window?

Original comment by mumu21...@gmail.com on 4 Sep 2009 at 1:57

[GoogleCodeExporter]

Personally I prefer the complete modified file(s), as I can then use Winmerge 
to show
and merge differences.

About your questions: It's really up to you, all options are open, I have no 
idea how
many individual file formats are going to be useful...
Your suggestion of using different extensions for different modes is fine, why 
not.
The only big constraint is that we can't put many entries in the file format 
filter
dropdown, for technical reasons. So if C64 should support many variants in 
loading
and saving, we can list "C64" as a single format (covering more than one file
extension if necessary), and if necessary we open a dedicated window with all 
the
special options when saving/loading: "[X] With separate palette", HiCol/MCol 
etc.
This solves the issue of auto-detection on loading. (CPC has same problem, 
files are
simply memory dumps without the palette and the mode). or as you suggested, 
force
different extensions to be interpreted as different modes.

Original comment by yrizoud on 4 Sep 2009 at 2:42

[GoogleCodeExporter]

I would think the when opening a *.C64 file it would detect what format it's in 
but
when you create a new image, I agree that there should be a "sub window" or 
dedicated
window to allow you to choose the specifics of the new *.C64. Now having said 
that, I
think it's important that all other formats that have specifics like *.C64 files
should keep the same type of sub window. I'm not even sure if there are any 
right
now, haha, but I do find it important. So, I assume that it's possible to 
detect what
format the picture is easily?

Original comment by thedaemo...@gmail.com on 4 Sep 2009 at 6:17

[GoogleCodeExporter]

For the pixel ratio, I think you have to call Set_video_mode to get the screen 
updated (this is what resizing the window does, basically).

For the patch format, I'm fine with svn diff, but the full files work well too. 
As 
you wish.

Original comment by pulkoma...@gmail.com on 5 Sep 2009 at 8:52

[GoogleCodeExporter]

Almost done now.
Some bugs in multicolor that I haven't solved.

However, on one of my computers it doesn't work at all, ubuntu 9.04/amd64, but 
on two
others it works, 8.10/amd64 and 9.10/i386
(I can test on 9.04/i386 and 9.04/ps3 too, if anyone's interested)
Looks like a bug in sdl in that ubuntu version. I have to investigate it some 
more.
Hard to know if my coding sucks or not, when the libs seem to behave in their 
very
own way.
Not completely sure if the multicolor bug is mine or belongs to sdl.

File format during save is decided from resolution (320 vs 160 pixels)
Then we could use some options, load adress, multiple files and so on. Is there 
a
framework for such option windows?

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 10:35

[GoogleCodeExporter]

Ok, here are the files.
However, saving fails if you are in magnifying mode, no idea why.
set_video_mode does not work, doesn't update pixel format until I manually 
resize my
window.
besides from that, it seems to work to load/save hires and multicolor pictures.
raw format, no loader address.

8000 bytes bitmap data 
1000 bytes screen memory
1000 bytes color memory
1 byte background color

hires pictures skip the last two sections, they know only about bitmap and 
screen.

At least this is a first version to start with, better to release this than 
nothing.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 11:42

Attachments:

• const.h
• loadsave.c

[GoogleCodeExporter]

Nothing special, they are window like any other. Basically :

Open_window(w,h,"c64 settings");
Window_set_normal_button(x,y,w,h,"text",1,1,SDLK_Return);
Update_window_area(0,0,Window_width,Window_height); // these vars are set by 
the 
windowing engine, don't replace them by actual values

do{
switch(Window_clicked_button)
{
case 0: break;
}
}while(1);

Close_window();
Display_cursor();

Look at any other window (most of them are in buttons.c) and at engine.h and 
windows.h for all the widgets we provide. Don't hesitate to copy&paste from 
other 
windows if you think they do what you want.

Original comment by pulkoma...@gmail.com on 5 Sep 2009 at 11:45

[GoogleCodeExporter]

Didn't see a problem when saving in magnified mode, can you provide some sample 
pictures so we can test more ?
Some bugs :
-I noticed if my picture don't match the constraints, I get a warning message 
but 
you still create a file. You should create it after having checked if you can 
save 
the picture, or delete it when there is an error.
-Watch out for whitespace. The usual setting is 4space per tab, indent with 
tabs. I 
also tend to use if (... instead of if( and avoid whitespace at end of line, 
and I 
also try to keep the code <80 columns wide, but this is not mandatory. At least 
fixing tabs would be nice, but I can do that automatically.

I'll let Yves have a look at the files before we commit them, and wait for your 
sample crashing pictures to see if I can help with that.

Original comment by pulkoma...@gmail.com on 5 Sep 2009 at 12:14

[GoogleCodeExporter]

The crashing seems related to some of the components in ubuntu 9.04
I get a feeling that Count_used_colors and Read_pixel_function (or whatever 
they are
called) gets the pixel data from what shown on screen, and not what's in the 
drawing
page/buffer.

I try to use tab with of 4, but some times I forget that when switching between
different computers.
But it looks like I have to stick to my asus eee 900, the only one at home with
working ubuntu....

About creating file before detecting errors, sure, I'll buffer everything before
write, it has disturbed me a lot too, as I get strange crashes on two of my
computers, leaving me with a corrupt file. Sometimes it doesn't complain, but 
saves
complete garbage anyway.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 12:22

[GoogleCodeExporter]

I just checked, you are using Count_used_color_area and this one is reading 
from 
screen instead of picture memory. This is obviously not what you want. This 
function 
is used to count colors used o screen by the preview thumbnail in the loadsave 
window. So you have to write your own function to check your blocks. Look at 
Count_used_colors to see how to do it properly. And feel free to rename 
functions if 
you find better names (could be Count_colors_in_screen_area() or something)

I'll check your other segfault bug meanwhile ...

Original comment by pulkoma...@gmail.com on 5 Sep 2009 at 12:30

[GoogleCodeExporter]

That makes some kind of sense.
I'll write my own that gets the pixels I want.

I can reproduce the segfault on the "working" computer too.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 12:47

[GoogleCodeExporter]

added
word Count_used_colors_screen_area(dword* usage, word start_x, word start_y, 
word
width, word height);

failed save does not destroy file.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 1:10

Attachments:

• misc.c
• misc.h
• loadsave.c

[GoogleCodeExporter]

and as a nice bonus, it seems to work on all my computers.
Now at least I have the tool I need to make graphics for my c64!

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 1:39

[GoogleCodeExporter]

Applied in r1013. I think you forgot to alter the old call to 
Count_colors_in_area 
to point to the new function in loadsave.c so I did it. Also translated some of 
the 
comments and added doxygen headers to the functions.

Can you provide some (links to) c64 picture files ? I'd like to include some in 
our 
pic-samples library so I can test things out. At least one for each format we 
handle. They don't need to be beautiful, they're just a way to check we don't 
break 
anything if a change is ever needed.

Original comment by pulkoma...@gmail.com on 5 Sep 2009 at 1:59

[GoogleCodeExporter]

It's not really a painter program format that exists in the wild, perhaps it's
compatible with koala.
This is the raw format every programmer wants for including the picture in a 
demo.
Perhaps with the addition to split the file in the 2 or 3 chunks, but a 
programmer
can do that himself, right? ;)
That will be in the next version, with settings window.

But ok, here you get two nasty pictures just to test that load and save works.
I guess I or someone else could provide better pictures some other time.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 4:09

Attachments:

• hires.c64
• multicolor.c64

[GoogleCodeExporter]

oups! you didn't get the latest version of my loadsave.c!
The one where I actually had renamed the calls to count_used....
And it's the one that doesn't trash files in case of error.
(the last one I uploaded, luckily I could get it back myself from this page)

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 4:19

[GoogleCodeExporter]

and there's another mistake, looks like you mixed up the count_used_colors_ 
functions.
I added a new one with a new one, but you seem to have renamed the new one, and 
added
my new with the old name.
Please check my files again before I do something stupid and go nuts again ;)

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 4:27

[GoogleCodeExporter]

Save options window almost done now, and loader improved to better distinguish
between different formats.
I also tagged the loadsaver as capable of comment, this seemed to be the only 
way to
show any "comment", which now instead shows what format it is, and the loader 
address
if present.
Is there a better way to do this, let me know!

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 6:57

[GoogleCodeExporter]

Instead of inventing new file formats, please existing ones:

http://codebase64.org/doku.php?id=base:c64_grafix_files_specs_list_v0.03

9002 byte Art Studio for hires (with .art as the extension), and Koala Painter 
for
multicolor (.koa) are probably the most compatible options. Make sure files are 
saved
with the load address, and not as raw files.

Original comment by MagerV...@gmail.com on 5 Sep 2009 at 8:38

[GoogleCodeExporter]

It's based on those formats.
And here is the loadsave.c with options to select load address and the 
possibility to
save bitmap, screen and color memory independently, with or without load 
address.

Original comment by mumu21...@gmail.com on 5 Sep 2009 at 8:52

Attachments:

• loadsave.c

[GoogleCodeExporter]

Ok, now the function names are reverted.

Original comment by mumu21...@gmail.com on 6 Sep 2009 at 8:51

Attachments:

• loadsave.c

[GoogleCodeExporter]

As of r1021, it is rather functional.
The window works but is a bit "rough", so once you're sure there are no more
parameters to add, we'll make a mockup of a prettier window, and then implement 
it.
In my opinion there's no way to make a unique generic window for 
c64/cpc/export...
lets make each format window unique and best-looking for its contents.

Your method to show sub-format information seems fine to me. There is too little
space in fileselector window to show it elsewhere; otherwise I'd have shown the
details for BMP and PNG sub-formats.

I can provide support for "multiple file extensions by format" for the 
fileselector,
so when "C64" is selected, it will show files with all extensions from the list
provided by MagerValp.

As Petter proposed in a revision comment, the save parameters should be 
replaced when
loading an image. The settings may be separate for Main and Spare - in case you 
load
2 images with different formats.

Maybe we can also make "Save as..." use the settings window, and "Save" skip it 
(if
some settings were already provided), I'm not sure.

Original comment by yrizoud on 7 Sep 2009 at 8:34

[GoogleCodeExporter]

good news!

At the moment I can't think of more parameters to add, but I've been wrong 
before.

multiple extensions is really good. I haven't checked how hardcoded the file
detection is. 100% extension? not checking file contents?

I have been thinking of that save/save as... too.

Original comment by mumu21...@gmail.com on 7 Sep 2009 at 8:41

[GoogleCodeExporter]

Detection relies entirely on on the various Test_*() functions, it tries all 
formats
until it finds one that works. With most formats, reading a few kb and matching 
a
header signature is enough. Checking file size is fast enough too.

The multiple file extensions is mostly so that I'll be able to make a default 
filter
"all images" that displays all files with extensions we can normally read, such 
as
.gif, .png, .koa, .art, etc...

Original comment by yrizoud on 7 Sep 2009 at 8:59

[GoogleCodeExporter]

While options are good, the defaults for saving really should be either a koala
multicolor, or a hires art studio picture. If you're saving individual parts of 
the
image (e.g. screen or colors), the extension should be .prg if there's a load
address, or .bin for raw data. This is the way other graphics editors work, and 
what
people expect. Pictures are either going to be standalone, in which case they're
compressed and display code is added (e.g. with
http://noname.c64.org/csdb/release/?id=76860 ), or they're going to be included 
in a
project, where again standard file formats are expected. Splitting is not 
needed, as
assemblers can include parts of a binary:

        .segment "GFX"

logo_bitmap:
        .incbin "logo.koa", 2, 8000

        .align 1024
logo_screen:
        .incbin "logo.koa", 8002, 1000

        .align 1024
logo_colors:
        .incbin "logo.koa", 9002, 1000

Original comment by MagerV...@gmail.com on 7 Sep 2009 at 8:17

[GoogleCodeExporter]

The only thing left to default is the load address then.
And the file extensions. But I guess you could save under any name you want?

Original comment by mumu21...@gmail.com on 8 Sep 2009 at 6:34

[GoogleCodeExporter]

Original comment by pulkoma...@gmail.com on 15 Sep 2009 at 7:13

• Added labels: Milestone-2.3

[GoogleCodeExporter]

I just found out that the hires load/save had the colors inverted.
Patch attached!

Original comment by mumu21...@gmail.com on 8 Mar 2010 at 10:07

Attachments:

• miscfileformats.c.patch

[GoogleCodeExporter]

(diff integrated in r1376)
I have noticed two problems in 2.2:
- Corrupted loading of 8000-byte hires files, because a 'break' is missing
- Many Multicolor files from random sources contain random "noise" in the high 
bits
of the the ScreenRAM ("nybble" block) and/or in the BG color. This causes the 
image
to get colors way above 16, and since the palette is normally not set there, the
image is not faithful.

I have pending fixes for both problems. Can't upload at the moment as it's a 
separate
branch where I'm doing FLI format loading (and hopefully saving, soon).

Original comment by yrizoud on 10 Mar 2010 at 1:56

[GoogleCodeExporter]

Original comment by pulkoma...@gmail.com on 5 Apr 2010 at 8:49

• Added labels: Component-Files

[GoogleCodeExporter]

Just found this : http://noname.c64.org/csdb/release/index.php?id=93314
May be useful.

Original comment by pulkoma...@gmail.com on 17 Aug 2010 at 12:12

[GoogleCodeExporter]

I'm starting to think we should allow a way to send the image to an external 
tool. This can serve either to perform a modification of the image (ex: running 
imagemagick), or any kind of export to formats that Grafx2 can't handle 
(Mufflon, or Magervalp's XI).
In the first case, Grafx2 would save the image as a temporary file, run the 
utility by passing the file name, and load the result as a new history step, 
without forgetting which "filename" it's normally editing.
In the second case, a temporary file would be saved too, and passed as argument 
to the tool; after running it would be deleted.

Lua scripts can act as wrappers to build the right command-line. And they can 
use input windows to ask for options, settings, etc.
The API can be something like: system(command) to run a command-line, 
name=savetemp(format) to make a file save that will be automatically deleted at 
end of script, loadtemp() to re-load a file that's modified.

Original comment by yrizoud on 17 Aug 2010 at 2:53

[GoogleCodeExporter]

Original comment by pulkoma...@gmail.com on 22 Aug 2010 at 1:34

• Added labels: Milestone-2.4

[GoogleCodeExporter]

The C64 stuff seems to work ok. Please open other issues if there are problems.

Original comment by pulkoma...@gmail.com on 18 Feb 2011 at 1:44

• Changed state: Fixed