stefanjl
commented
7 years ago Ok... i see now when i have access to the new AL design that it is basically much like the old AL structure wise, and my suggestions is a big change. What is your opinion?
The detailed information for the game structure i suggest is pretty much based on Atarimanias since thats what i am used to. For example the structure of AM is allowing the ability to specify wich release that is STE compatible... in Dragon Ninjas case so does not the first release suppert STE but the later budget and compilations release do support STE... this goes for several other games also.
ATM so have i not been able to see how a game entry might look at the new AL, but i have been told they can be found here at github though, just have not found it yet and i will search after this post :)
I am very much for documenting as accurate and detailed information as possible for all official Atari ST game releases. My first experience with game preservation and documentation was the MAME project wich i helped by dumping roms and searching undumped arcade games, you could say the MAME team are my teachers :)
stgraveyard
stsilversurfer
Brume-AL
Hello @Brume-AL , @stsilversurfer ,
For the past days I have been talking to Stefan Lindberg of Atari Mania. He seems like a very dedicated guy. He has been sharing info regarding the setup of the Atari Mania game section and I have learned a lot and it seems to me our game sections is missing some really crucial stuff regarding game versions. I will copy/past our complete conversation. This is a lot of info, please read it carefully when you have time. I think we do need to make some changes in the near future to the game structure, but this is something we should analyse and think about very deeply before continueing.
Here goes :
STEFAN wrote :
Hi,
you asked me to send an PM with my questions :) As you might know so am i already part of the Atarimania team and have been for 3-4 years now and i have asking for updates at the AM admin forums to more suit what need to be documented and how, but there is only one person that have access to actually update AM design/layout and he is practically not present at all and my discussions at the forums usually only gets replies from Marakatti and he can do nothing but saying he hope Franck will see the discussion and reply, and i do very rarely get any reply at all. This got me very worried when only one person controls a database and if he disapears so does the database (like Guardians of the Past) and in early 2015 i had the idea to get new ST database in works and i got my brother to promise me to program it (i have no programming skills) and then i started looking for people to be part of this new database and i noticed Brume had been posting lot's of posts here at Atari-forum about all ST original games he had and i thought it would be a good idea to contact him and ask if he would like to join me in starting a new ST database... but it turned out the new Atarilegen was just in early works and instead Brume asked me to join the new Atarilegend in early 2015, he welcomed me to AL team and said he would give me access to a private forum to propose my ideas. But i never heard from him again despite send a reminder PM. I know he is very busy and maybe he has to much to think of. And after taht so was i restrictive with adding info to AM database since i thought the new AL was about to be online soon and i did not want to add the info twice, but now i am adding lot's of stuff to AM and i once again get frustrated on how difficult it is to document all info i want. Some ideas i had is already part of the new AL... like having crack menus as download, thats a very good idea since crackers often got hold of early versions of games that might be difficult to get hold of today... for example the "Cisco Heat" thread i posted here at AF because i had seen the screenshot at the AM database was different from the one in the game i played, it turned out the screenshot for the AM Cisco Heat entry was from a cracked version and if i rememeber correctly so was it from a Pompey Pirate menu and i suspect they got hold of an review copy that they cracked, this is quite valuable information and the PP crack might be the only thing left of that early version of Cisco Heat.
other idea i have is to have screenshots with comments just like Moby Games have, and another much more controversial idea is to have screenshots with the border visible, to me the border is very much part of the Atari ST experience... most of the time it is of course black but several games use other colours, i suggested the border idea at AM admin forums and even said i myself would redo all sceenshots with border :) It was not an idea that got much understanding :) But if the screenshots always were at the same resolution then i proposed if the original screenshots were with border but the database system uatomatically cropped them to 320x200 and then let the user/viewer of the database to see the borderless or the full border version of the screenshot... i still think that is a very good idea... maybe a cookie stores the info if screenshots is to be seen with or without border. Anyway nowadays i always upload new screenshots with border to AM :) example: http://www.atarimania.com/game-atari-st-formula-1-grand-prix_9370.html
Ok, now more difficult things and this is more what i was asking in the forum post you replied to... how should the game releases/versions be presented? Will the releases and versions have their own entry so to speak like Atarimania? Or just mixed together like for example HOL (amiga) database? This is actually a strong plus for the AM database that it allows for that kind of seperation even if it needs to be updated for better flexibility. I always like the idea to have first a main entry for the game with mixed info from the various releases just like HOL... but then also have the ability to in detail check out the sub-entrys of the game like budget release, preview realease, compilaton release, different country release. This was something i wanted to discuss when the new AL was already in early stages but i suspect it is too late now... or? I know what i want when i am researching Atari ST games, how to find new software versions... releases with different packaging and such. I let you read this before writing more just so i know your standing in all this? Also i bet there is an limit on how text a PM can contain :wink:
Stefan Lindberg
Maarten replied :
Hello Stefan,
Thank you for your long pm. It shows clearly the frustration you, as a DB guy, have and this is a bit the reason why I started working on AL again and in the way I would like to move forward now.
I will try and answer everything as detailed as possible.
Exactly, the same goes for the story about Atari Legend. MugUk, Marcer, Champions and so many others have spent ours updating the DB, yet from 2008-2009 the project was abandonned because the devs (me) just didn't have the urge any more to work on it. So in that respect I do really understand Franck. Working on these projects takes a lot of time (I don't need to tell you this of course), I have been working for 2 years now on the new AL. That is why I want to open everything up. Create API's, downloadable database content, technical document regarding the public, release the complete source code ... just everything. This way, it becomes more something for the scene and if at one point in time I want to take a break, everything is there for every one. I am so happy that all the work that went into AL in the past, will now be used again, even if it isn't as complete as Atari Mania. But that would have never happened if I did start working on the project again and I don't like this way of thinking. Also, regarding Guardians of the past...These kind of projects can be revived again, if they use one central DB (example the new AL db). I want everything to be usable for everyone and vice versa!
All I can say, you are welcome to the team if you want. I am really looking for motivated people with new and good ideas. Regarding your DB idea, well, the AL database currently contains over 170 tables, all normalised and linked together. I'm making the project bigger and bigger. I still need to document it, but I'm doing it in such a way, using schema's and examples, that non technical people can understand it as well. I think you would be amazed. So in that perspective, I/WE are building what you wanted to make. But it is also the scene that needs to accept and embrace this project, because I can not do it all on my own. We currently have about 8 people in the team, but most of them are a bit in active and doing other stuff like the whole disk archiving. I'm currently doing all the coding on my own. That is why I want to release something in 1.5 months so the public can finally see what I am doing and I hope to create interest on all levels. Please continue working on AM, as this is the best game db around and I am not planning to do that work again. I'm still trying to figure out a solution to open all the data, but it is not in my power of course. Currently, I'm really busy for the release, but when it is done, I would gladly do a demo for you, ecplain the DB, explain the structure, welcome you to the team, explain the new forum so we can share ideas! Everything you want...
First things first. It is never to late Stefan! ;-) It is never too late to change the db if we can perfect its structure, all we need to do is convert the data. But that is always doable if we take our time, do a decent analysis and use our heads. No manual work is needed, we write scripts for all this stuff.
I am gonna try and answer your question to the best of my knowledge. In the beginning of this year, I did a new analysis together with Brume and ICS regarding the downloads structure of the new AL project. I've always been a gamer and I don't have the deeper knowledge of different downloads, cracks, hacks, menus ... like you, or Brume or Makaratti has, that is why it is important to talk, discuss and make analysis together.
Everything is very modular based on DB level and on project level. We have a game entry in the db. A game can have a different name, it can be an official release, it can be a budget release ... and so on and so on ... like at AM I suppose. However, we now have created a very elaborate downloads table structure, meaning a game can have all kinds of downloads versions. It can be an official download, a crack download by a single person (coming from a menu - and thus linked to the menu section and to the individuals section (crackers), a menu download ... Seriously, like I said it has been a few months now so I need to read my analysis again. But it is safe to say, the structure is already very deep, the options are fantastic (as Brume wants to build something amazing, with these kind of circular schema's on how valid the download is and stuff like that). When you see the structure and you have ideas, because everything has been modularly build, new tables can be added and linked to the download structure without problems. IT goes really really far. We also have a complete 'MENU' structure ready which is complete bonkers :-) as well as a scene structure for crack and hack teams and everything. For all of these structures, the control panel is ready! however, the actual site isn't yet and the tables are still empty. In a new fase, I will be thinking of ways to fill the db from Stonish, from the scene archiving team, from Fujiology ... In ways we will need as less efforts as possible ...
Regarding the screenshots, you are correct, you don't need to make these again. you can write code to make borders. I had this fun idea to give people the chance to make screenshots go in 'st low' with an interlace effect and in ST mono, stuff like that. but I have put this fun stuff aside as I don't have time, first things first and that is the basic site :-) Auto cropping is no problem as we already do this, I have writting a script for that.
If you have more questions, don't hesitate to contact me.
Cheers, Maarten
STEFAN replied :
Hi Maarten and thanks for your reply :)
First i think you misunderstod about screenshots with border, i always want to document as detailed as possible and having the borders as part of the screenshots made for AL is a way to document the game... many games use different colour for the borders and many other just use black and sometimes you get rasters in the border area...and i have an example of multicolour border with raster effect and thats Wrath of the Demon, check out the following link to see my video on youtube and you see the part of the game use the rasters: https://youtu.be/-1qTULa-g4A?t=24m54s
I did not mean borders should be added automatically by the DB system... they should be part of the screenshot from the beginning.
I do know people don't want to see borders in games or screenshots but i feel the urge to document that.. sorry :) Would you at leat have the option for border screenshots to be uploaded to the DB and then autocropped by the system to 300x200 and then two screenshots is stored in the DB and the viewer can select if border screenshots should be viewed or the borderless one. I think using Steem SSE option "border:large" would be useable and it saves the screenshots in 416x275 reolution.
And another thing that i actually suggested at AM and it was added by Franck is to use PNG, actually i think only PNG should be used for screenshots at AL... jpg add compression artifacts... gif is limited to only 256 colours... and also PNG is a free format :)
So much to discuss about even simple things like screenshots :)
More about the layout and game entry. At Atarimania so is the games seperated by publisher (and then again by country or distributor) and i guess it is the best way to use... i don't really have any other ideas how it should be made if not that way... this was one thing i always would have liked to discuss with other people how it should be done.
I use the AM and even HOL and moby games for research about different verisons of games and such. And now i come into screenshot subject again... something i want when research about games is that the screenshots specify wich version of the game they come from... once again see the Cisco Heat example i mentioned in the previous PM, the AM entry for Cisco Heat was not using screenshots from the actual dump that was downloadable from the entry... so in AM case so would it be very possible to have screenshots exclusiv from that specific entry (if downlaod is available) but most of the time so is the same screenshots used for all releases of the game (not talking about Cisco Heat in that case as it only had one official release)... and i think it is like that just so some entrys with no dump available would have no screenshots and then when a dump becomes available so is the screenshots not changed.
So for example a game is first released as full price in UK, France and Germany, this would be one entry on the DB and the are seperated later into sub-entrys (or what to call it... i am no DB coder) by country... in AM so do they have seperate pages when browsing, and when in this way i would like that each page showing the country version use screenshots from a dump that comes from the country release they show... like browsing the page with the german release shows screenshots taken from the german release dump and not the UK release even if the game is still in english for the german release... this is just so the actual entrys can be useful for research and documenting the history of Atari ST games in detail. the same game might get an budget release later on... lets say Kixx label and then of course that entry will have screenshots from a dump of the Kixx release and not the first fullprice, but of course if no dump is available then maybe a place holder pic from the fullprice release can be used instead of nothing.
Now i have been using screenshots a lots as a way to explain how i use databases but the stuff i said about screenshots go for dumps also... for example dumps of the UK release should not be used as download for the german release even if the game for both releases use english language and looks identical. This means contributing dumpers should be as detailed as possible when contributing games and alawys supply a photo of the games disk(s) and/or game box... much like i do when i post dumps here at AF's SCPro forum... see example:
Everyone these days have access to a digital camera (all mobile phones have it) so there is no excuse not to include it together with the dump.
I hope this was not too confusing... i am quite good at english but no expert in writing in it as it is a foreign language to me :)
And then there is how the pirate menus should be presented in the DB? how is it solved now? And all those HD patched games like from Atarizoll (ppera).
Stefan Lindberg
Maarten replied :
Hello Stefan,
It is clear to me that I could really use someone like you to help think about the db structures, too bad you weren't there 2 years ago. You do give me a headache for sure, because what you propose here goes really, really deep imo. But than again, I am no archivist or db guy, so ... read on ...
AL only uses png for screenshots. So, if I understand correctly, you are not only talking about a physical screenshot, but also about preserving all the info regarding the borders (which you think you could explain by use of screenshots) , right? So from a technical DB pov I need to ask the question (let's do this exercise as an example), the difference in borders, is this per game, or can a single game have different versions with different border styles? If the answer is NO, we could link 1 or more tables containing to the game main table, like a game_border table. This table contains all kinds of info regarding the border (the technical stuff you mention above) and this table could also be linked to the screenshot table, so you have the chance to add a screenshot with the special borders. That could be a very nice and modular solution. If a game has more border types by release or download, these tables could be linked to the download tables. You see, such an addtion to the existing tables structures is easy, as long as you made a well thought out structure (which I thought we already had - but after reading your email, I see we are lacking in some parts - read on).
Ok, first off, I would really like to discuss our complete game structure with you at one point in time. Our game structure has mostly stayed the same as AL1.0, and I think the game structure of AM might be better, I don't know. It is impossible to discuss about this in a pm, without the DB schema's but I do have some remarks and thoughts. When I look at AM for Cisco Heat, I see 1 entry. However, the game has different countries. Compare this to the way 'Secret of Monkey Island' is setup at AM, this has an entry for each country version. This doesn't seem uniform to me, as it seems in the AM DB, Secret of Monkey island is setup per country, while cisco heat isn't.
Now, over in the AL DB, we create a game, we link as many publishers and developers and authors to this game. We are also able to link a country and extra info to a publisher and developer. Example, Cisco heat can be released by image works as a main release in the USA, but it can also be released by Renegade as a budget title in the UK. The same goes for release year, extra info (like budget, main release...) can be linked to the year. However, the fact remains that all this info is mixed together and linked to 1 main game entry!!!! And I think the same is done over at AM, and that you guys "solve" this by adding a game more times in the DB with different info (which is an ok solution, but this could be made better on a db level - this is something I would definitely want to think about with your AFTER the first release, if you and others are interested).
HOWEVER, now for the interesting part. When you talk about your versions and screenshots of a game, you are always talking about it together with a DUMP of that game. So in other words, you want to add these screenshot types and other info to a download. This is another way of thinking about it. We over at AL have a very deep downloads structure. So a download is linked to a game. This download (or dump as you call it) can have its own language, its own county, its own type (is it official, a crack from a menu...). So you see, it is again a bit the same as the BORDER story above, at what level do you link this in the structure.
I absolutely have NO concerns regarding the border and dump story you are telling me here. These things are additional information and can be added to our existing structure in a very coherent way, which I would gladly provide (however, how we will get all the info (from AM???) or who will fill all the tables - That is another story!). What does concern me is what you have told regarding the different country versions! This is something that doesn't seem completely right at AL, nor at AM. I think this could be made better, because the way I see it now, at AM you have all this info regarding countries mixed together linked to one game, or you have multiple DB entries of the same game (like monkey island), which is also not 100% correct from a DB analysis perspective (as you create redundant data). Maybe I am wrong here as I can not see the DB structures of AM, but I think I am correct.
I would really like to copy our discussion to our AL dev forum so Bruno, MArcer, ICS and the others can read and share their opinion on this, so maybe I can do small conversions to the DB at game level to make it better. But more thought and analysis are needed.
1 concern remains Stefan. I am not a fan of doing a job twice. So in the end, I can build all this great stuff, my heart would break if we need to do all the work, that has been done over at AM again in a new DB structure of AL. The reason I want the DB to be open to everyone is just to avoid this kind of stuff. So this is something we also need to think about. You are saying you also use moby games for research. I know Moby is developing an API, which gives developers the oportunity to link to moby to fill their DB. This is what I will do in the future release, so no manual work is needed. I want to do this with other projects as well, to build the ultimate open source ST DB for the scene.
We need a guy like you on the dev board, and I think it would be good if you also learn a bit about DB structures so you can participate on a more technical level on the discussions. I can always learn you a bit by use of schematics I have made, it will be much clearer for you when you see these things...
If you have more remarks, or if I have misinterpreted some of your info again, let me know ... keep these questions coming ;-)
Cheers, Maarten
STEFAN Replied :
Hello Maarten :)
You are correct it is difficult to discuss
What you said about screenshots is exactly what i ment but i did not know how to explain it in a good way but your words are excellent when you said i would like to document the borders by using screenshots :) And i would like that screenshots should always be made and uploaded to the DB with borders as standard. I dont know if there is a game with differnt verison that use different border... but thats something that could be discovered if screenshots with borders are used.
About Cisco Heat and Monkey Island... First have in mind that all info at AM is not correct, it gets updated all the time :) I belive Cisco Heat was only released in english by the same publisher and thus gets one entry in the database while Monkey Island have specific version for other countries like France for example in wich the gamebox is is in french and manual in french and the game is in french.
I like how AM is structured... but it is not perfect and can get better but it is the best one available right now. AM is the best game DB on the internet for any system... HOL and Moby Games have no chance compared to it, the main reason i use HOL for research is because it lists compiliations in the entry so you know wich compilation the game was used in since most of the time so was the same compilation released for both Amiga and Atari ST... sadly so does not Atarimania list compilations in game entries, i have suggested it many years ago but Franck has not yet made an update to support it.
About "dumps"... i use the word "dump" becuase it is easier to write than the word "downlaod" :wink:
In AM DB so are a game first added with basic information and such but do not get an entry until you add an "publisher", lets take "Bad Duded vs Dragon Ninja" as an example... if you serach "Dragon Ninja" at AM you get 2 hits... one from publisher Imagine and one for the budget release Hit Squad. And in the Imagine entry so can you find scans of the Imagine gamebox and a downlaod that is from the Imagine release of the game and in the Hit Squad entry so can you find scans of the Hit Squad box and downloads of the game disk taht is from the Hit Squad release. And this is good way to seperate different releases and downloads... and if i get my wish fullfilled so should these two entries use screenshots taken from the download on each page.... like the Imagine entry use screenshots taken from the Imagine download and the Hit Squad use screenshots from the Hit Squad release. And of course the downlaods used in these entries must come from the particular release, and as i understand so are downloads clearly seperated in the new AL so that should not be a problem in case the new AL begin to use a similar structure as AM.
I was thinking if i should show you the interface we use at Atarimania when adding games and edit the database.. i mean the one Franck supplied to us moderators, do you think it is ok if i show it to you? screenshot of it or video.
Stefan Lindberg
MAARTEN replied :
Thank you for all this information. Just talking about this has given me food for thought, so to speak. I have learned a lot. At first sight, I really like the way AM is setup. You create a main game and it gets online when it is linked to a publisher. It seems very well thought out. I wonder how this looks on a DB level.
Stefan, I hope you don't mind, I want to share this conversation with the AL team, see what they think and how we could expand and make our game structure better. I really like the job we are doing, but it seems to me the way we handle different releases is not good.
This all being said, the main problem stays, as expected. I can put all this effort in building the ultimate Atari ST database (and I am happy I can expand from what we had), the fact of the matter is that I'm am doing work, that is already done. This is a bit stupid, but I don't know how else to solve it. I think there should be an open source Atari ST database! So I'm building it as part of the AL project.
I just wish that we were all working on the same project ...
Anyhow, I'm very interested in seeing the AM admin pages, but I don't want to look like I'm spying. I am all for complete transparancy, but of course, not everybody thinks the way I think. So if it is up to me, you can show me the AM backside, but I don't want anyone to get in trouble.
Like I said, you are also welcome to the AL team, to help think about db structures and ways to AUTOMATICALLY fill the database (we don't want to do manual work that has already been done in the past). And of course, you can stay on the AM team simultaniously.
For now, I still need to code. I just want to release some stuff soon so the work we have been doing in the past 2 years can finally be shared with the public.
Cheers, Maarten