Obsolete Url And Version Legacy In TripleA Maps Yaml And Versioning Practices

I've seen that in triplea_maps.yaml there are still the urls referencing to the releases.

What's the point of that? Shouldn't they be substitute with a generic reference to the repository?

For example, with the latest 2.1.0 update, @ron-murhammer changed the url from https://github.com/triplea-maps/world_at_war/releases/download/0.23/world_at_war.zip to https://github.com/triplea-maps/world_at_war/releases/download/0.25/world_at_war.zip .

What's the point of that, since the Download Maps will have you downloading: https://github.com/triplea-maps/world_at_war/archive/master.zip anyway?

Generally, I tend to think that the best system would be that you automatically download the latest release, if so specified in the url, or you can be prompted to download a specific release (that may not be the latest), if so specified in the url. If this is not feasible, then the current behaviour of always downloading the latest, without requiring updating the triplea_maps.yaml each time, is preferable.

Moreover, if now I download world_at_war, the properties (world_at_war-master.zip.properties) are:

Sun Sep 18 09:36:15 CEST 2016

download.time=Sun Sep 18 09:36:15 CEST 2016 map.url=https://github.com/triplea-maps/world_at_war/archive/master.zip engine.version=1.9 map.version=1.0

How will I be told when a new version is out, if the version seems always being 1.0? What is the point of having that?

So, as I've pointed out, there are two issues with this system:

1- You still have the reference to an old "releases/download", thus you feel prompted to update that, for no apparent (to me) reasons.

2- You still have to update the "version" in the triplea_maps.yaml, which practically obliges you to update triple_maps.yaml each time anyway, not to have a false version info upon downloading.

The first issue should be solved as to have a generic url, since there is no point in updating it anyway, because the current system induces you to update it regardless, to not have an obsolete reference, even if it is currently pointless, since you download the master, anyway, which makes no sense.

The second issue should be solved by deleting the "version" or having it updating automatically or having a version.txt or something for the whole map (not the single xml in it) integrated in the map's folder, otherwise this would practically frustrate not having to update triplea_maps.yaml each time, since you have to update the version each time anyway, not to give the users a false info. Notice that the version is not supposed to be the same as the one of an xml inside the map's folder, especially in the cases of map's folders with multiple xml, each one with its own version number. It is actually highly advisable the download version not being supposed to be the same as the version of some xml inside. For example, what about if ice wants to upload his WAW40 mod to a new version? In this case, the user should be get told that a new version of "world_at_war" is disposable, despite "World_At_War.xml" and anything buth the ice's mod being the same. Should the downlad version be updated to something more than 2.1.0, despite the fact that World_At_War.xml is still at 2.1.0? It is thus advisable the download version being unrelated to any xml versions the map's folder may contain. Moreover, also the mapskins (not having any xml inside, but still due to be updated) and the maps containing only disparate mods, referencing to other maps folder, need versioning; thus such versioning should a harmonised accordingly, amongst all these differing items. Thus, I believe that also the current practice of having the download version being the same of some xml version in the package you are downloading is bad. The download version should be just an incremental numeral for the actual progressive releases of that map, like it is now the case for the "releases/download" numbers. A good practice would be that the download version is the same as the releases/download one. Thus "world_at_war", as a download, should be currently at version 25, not 2.1.0, as that is just the version of the World_At_War.xml, but you may be updating some other xml or something not xml related, like a wrongly named unit (marine instead of Marine), without making any sense to modify the game's version inside the xml at all, which is something you should change only when anything at all has changed for the xml itself. Currently, in "Download Maps", everything is given as being "v1.0"; what's even the point of that. Rather better not showing this up at all, if they have to be all "v1.0"? Since the download version number, referenced in the related downloaded properties, used to be relevant only to tell the user if the map he eventually possesses it's outdated, and a more recent release of the same is available for integrated download, I suggest you somehow make sure this function is still disposable, without needing updating the triplea_maps.yaml each time, that would practically frustrate the benefits of automatically downloading the latest version (via downloading the master) or, worst, create confusions as maps being sometimes updated without the users receiving the info about that.

My suggestions, if feasible are:

1- Substitute the url releases/download reference with a generic reference (maybe the reference to the "archive/master.zip" itself).

2- Having the Download version number automatically updating to the latest releases number (thus, for example, world_at_war should be currently version number 0.25) or anyway having a way to tell that the repository has been updated, without requiring to change the triplea_maps.yaml (as that would either frustrate the advantage of not having to update the yaml at each new release, if the mapmaker is supposed to update the download version number, or, possibly worse, having no actual relation between actual updates and versioning updating, which would be confusing).

Also a question is how does the user get told that a new version is available for integrated download, under the current system? I don't understand how it works now.

Why did @ron-murhammer updated the url and and the version at all, after releasing world_at_war 2.1.0? What is that doing at all?? Isn't the master now downloaded each time, anyway, and always at v1.0???

Otherwise, what is the point of being exempted from having to updated the triplea_maps.yaml each time you update the referring repository, if this just means that obsolete info are being kept there or the user is not informed about a new version being available for integrated download?

Please, tell me what I'm missing and, for the remaining part, I think all the above issues should be sorted out (and documented) before the next release, not to confuse people downloading maps under the new system or mapmakers updating their work under the same.

For example, with the latest 2.1.0 update, @ron-murhammer changed the url from https://github.com/triplea-maps/world_at_war/releases/download/0.23/world_at_war.zip to https://github.com/triplea-maps/world_at_war/releases/download/0.25/world_at_war.zip

Where was that update done?

How will I be told when a new version is out, if the version seems always being 1.0? What is the point of having that?

The version does not have to be pinned to 1.0. . When it is incremented to '2' in the triplea_maps.yaml file, then the game will be able to determine if your map is out of date.

1- You still have the reference to an old "releases/download", thus you feel prompted to update that, for no apparent (to me) reasons.

Where is this? What do you mean by "feel" prompted?

2- You still have to update the "version" in the triplea_maps.yaml, which practically obliges you to update triple_maps.yaml each time anyway, not to have a false version info upon downloading.

Yes, automatic update of the version in triplea_maps.yaml would be nice. It's not clear how to do so deterministically, I'm not sure if we really want to update it for every little minor change.

Thus, I believe that also the current practice of having the download version being the same of some xml version in the package you are downloading is bad.

That is no longer the current practice. The download version is in triplea_maps.yaml, and you've pointed out how it is version '1' for all maps now.

"triplea_maps.yaml each time, that would practically frustrate the benefits of automatically downloading the latest version (via downloading the master) or, worst, create confusions as maps being sometimes updated without the users receiving the info about that."

There is no current requirement to update the version for every change you make to a map. IMO at minimum it needs to be updated when compatibility breaking changes are added, otherwise to some extent it is up to the map maker when they want to flag there is a new version available. Perhaps that will be on every update, perhaps not. I don't think we can really know exactly what is best there until we gain some more experience with the updated map distribution process.

I think it would be great to have all download version information stored with the map, and not centralized to the triple_maps.yaml file. On the other hand, we've already had some drastic changes, and at some point we do have to start using the new map process so some of the confusion can die down.

"1- Substitute the url releases/download reference with a generic reference (maybe the reference to the "archive/master.zip" itself)"

Unless some URLs were missed in triplea_maps.yaml, this was already done

@Cernelius I believe the URL update isn't needed any more and I just did it out of habit.

@DanVanAtta How exactly are we triggering that a new version of a map is available since we aren't pointing to the specific map version URL anymore? It also currently seems to be showing that every map is at v1.0 even though triplea_maps.yaml shows differently? There seems to be a lack of clarity on how maps are being versioned and potentially old fields that aren't being used now?

@DanVanAtta ... and what is the relation between those two versions of triplea_maps.yaml, one in

https://github.com/triplea-game/triplea/blob/master/triplea_maps.yaml

and one in

https://github.com/triplea-maps/Project/blob/master/production_config/triplea_maps.yaml

@DanVanAtta How exactly are we triggering that a new version of a map is available since we aren't pointing to the specific map version URL anymore?

Note the 'version' in the old XML spec, for example:

$ curl http://pilotfiber.dl.sourceforge.net/project/tripleamaps/xml/category1VeqrynDepot.xml
  <game>
    <url>http://downloads.sourceforge.net/project/tripleamaps/maps/Diplomacy.zip</url>
    <mapName>Diplomacy</mapName>
    <description><![CDATA[
    <img src="http://tripleamaps.sourceforge.net/images/TripleA_diplomacy_mini.png" />
    <br>An adaptation of the game "Diplomacy" for TripleA.
    <br>Includes 3 Free-For-All (ffa) games, and 1 normal game:
    <br>
    <br>1) Diplomacy
    <br>This is as close as TripleA lets us come to the real Diplomacy board game. Uses some complex rules, be sure to read the game notes.
    <br>
    <br>2) Diplomacy: FFA V3 Rules
    <br>This is the Diplomacy map of Europe, with ww2v3 units and rules.  Easy to learn.
    <br>
    <br>3) Diplomacy: FFA Great War Style
    <br>This is the Diplomacy map of Europe, with "Great War" units.  Also very easy to learn.
    <br>
    <br>4) Diplomacy: WW1
    <br>Unlike the other 3, this is a normal non-ffa game.  It pits Germany, Austria, and Turkey, against France, England, and Russia.  (Italy is neutral)  
        It uses Diplomacy style units, Armies and Navies, with 3 territory types: Land, Sea, and Coastal.
    <br>
    <br>
    <br>Be sure to download some of the map skins for this game.
    <br>
    <br>by Veqryn
    <br>With help from Pulicat and Bung
    ]]></description>
    <version>2.2</version>
  </game>

Please note the: <version>2.2</version> That has been the trigger for whether maps are out of date. It never had any relationship to the version in the map XML and is only used for the 'out of dates' map feature.

The YAML format conversion was a 1:1 update w.r.t properties. The version field was carried over exactly.

The version number in this file was already pretty confused. It looked like the expectation was for it to control map compatibility, when it did not. Hence it often tracked a map's version as found in the map game xml. Except, this strategy makes no sense when there are multiple games xmls with different numbers. That is not to mention that this version number never controlled map compatiblitiy in the first place, just only to indicate to the game engine if a map is out of date.

Hence this snippet from the .properties file is how the game sees local map download versions:

#Sun Sep 18 09:36:15 CEST 2016
download.time=Sun Sep 18 09:36:15 CEST 2016
map.url=https://github.com/triplea-maps/world_at_war/archive/master.zip
engine.version=1.9
map.version=1.0

As much as I can tell, only the 'map.version' field is used, but it is that version field that maps against what is shown as available in the triplea_maps.yaml: https://github.com/triplea-game/triplea/blob/master/triplea_maps.yaml

So to answer this question directly "How exactly are we triggering that a new version of a map is available since we aren't pointing to the specific map version URL anymore?"

That is done by updating the version field in: https://github.com/triplea-game/triplea/blob/master/triplea_maps.yaml For in-game compatibility, you have to still update the version attribute in the 'game' XML file.

@DanVanAtta Yeah, sorry I meant the version field not the URL. As I didn't realize we had moved the triplea_maps.yaml file over to the engine repo.

Are we looking to eventually delete the maps Project repo all together? Why move the config file back to the code repo rather than in a maps repo? I think there is a good bit of confusion on what the process is here and what the vision for it is. Do we have the latest map process documented?

Are we looking to eventually delete the maps Project repo all together?

I think it should be evaluated. For now it seems like we are more scattered with yet another issue queue in that project. Perhaps it'll be easiest for map global problems to be reported here. Second, most of the scripts there are not as necessary anymore. There is not much content to house there. Last, the documentation stored there scattered the docs.

Why move the config file back to the code repo rather than in a maps repo?

There was not good visibility on the changes to it in the maps repo

It scattered the 'production' files read by the game, all of them except for the maps file are in the game engine repo.

Allows for travis to publish a copy of it to the website.

Do we have the latest map process documented?

For the most part. This is another whack at the docs: https://github.com/triplea-game/triplea/pull/1240 At this point it really needs to be updated further by someone that is not me. I have too much context to know what is not clear at this point. Alternatively need some solid feedback on what is needed as far as docs, and what is confusing.

@Cernelius I believe this should answer all the questions you had, let me know if there are any outstanding questions:

2- You still have to update the "version" in the triplea_maps.yaml, which practically obliges you to update triple_maps.yaml each time anyway, not to have a false version info upon downloading.

The false version is not a problem. The version is only used for determining if you need to re-download the map. If you update the map and do not increment it, it is basically saying that is the same version as the last one deployed and the two will both work together.

The first issue should be solved as to have a generic url, since there is no point in updating it anyway, because the current system induces you to update it regardless, to not have an obsolete reference, even if it is currently pointless, since you download the master, anyway, which makes no sense.

There is confusion here since the masters are actually coming from the URL, you were going off of an old maps.yaml file.

It is actually highly advisable the download version not being supposed to be the same as the version of some xml inside.

Agree, that is the case. The download version has been reset to '1' since it was inconsistent and all over the place. At this time since we are resetting the maps, '1' was an easy new starting point.

Moreover, also the mapskins (not having any xml inside, but still due to be updated) and the maps containing only disparate mods, referencing to other maps folder, need versioning; thus such versioning should a harmonised accordingly, amongst all these differing items.

I tend to agree, though to the extent we have harmonised versioning I do not think was ever built into the system.

The download version should be just an incremental numeral for the actual progressive releases of that map, like it is now the case for the "releases/download" numbers.

Yes, intended to be the case now as described : )

A good practice would be that the download version is the same as the releases

The releases are generated by Travis. We are cutting that out for the moment.. It's quite the step to get that working, and essentially I'm the only person who is going to be doing it. Bad sign for there to be such a process bottleneck. I'm not in love with the decision to drop Travis, but it has simplified the map repo setup tremendously. Anyways,my point here is that brand new maps are not going to have any releases at all, and we may decide to turn it off completely since we're not really using the released version anymore at all.

Since the download version number, referenced in the related downloaded properties, used to be relevant only to tell the user if the map he eventually possesses it's outdated,

Still is used for the same.

1- Substitute the url releases/download reference with a generic reference (maybe the reference to the "archive/master.zip" itself).

Done : )

2- Having the Download version number automatically updating to the latest releases number (thus, for example, world_at_war should be currently version number 0.25) or anyway having a way to tell that the repository has been updated, without requiring to change the triplea_maps.yaml

This would be pretty nice. But, it's hard to create a rule to know when to always do this. Otherwise, it's a level of automation that is maybe feasible by having the game check each map repo for the time of the last update. I've added a note for this on the feature backlog list: https://github.com/triplea-game/triplea/wiki/Feature-Back-Log

Also a question is how does the user get told that a new version is available for integrated download, under the current system? I don't understand how it works now.

https://github.com/triplea-game/triplea/tree/master/docs/map_making was updated to answer that question. Please submit a new issue to give feedback on that page, or do direct edits to give more direct feedback (the latter being most preferred)

This is just a confusing and bloated discussion that arose from me and redrum not noticing the new yaml, thus talking about an already obsolete item; so it deserves to be closed and eventually resummarised anyway, because it is mostly pointless.

I think I agree with everything you said here, @DanVanAtta , but I'm unsure about several items.

In particular, I think I fully agree with this too, yet I'm not sure:

I think it would be great to have all download version information stored with the map, and not centralized to the triple_maps.yaml file.

For the release number, I think the best would be either:

automatically updating to the number of releases. Like "world_at_war" should then be at "14".

having "X.Y" number, with "Y" being increased at every new release keeping compatibility and "X" being increased at every new release breaking compatibility. Like, world_at_war should then currently be "3.1".

However, I can easily see mapmakers and developers derail from the second system, and go back to equaling the download and xml versions (like having WAW at 2.1 instead of 3.1, despite this being the second time we break backward compatibility), which is especially nonsensical for maps (like WAW) having multiple xml of different makers (Sieg and ice) inside; thus I feel strongly inclined to suggest a fully automated versioning system, even tho differentiating between compatibility breaking changes and not would be neat, but not that needed. Also there are some grey areas, where a map would not crash, and would be theorically playable, but would make no sense anymore, plus there is the item of forward compatibility, where I could specify some things in the skin, currently unused but usable in future xml versions (for example, I add Yunnan beside Upper Burma in all txt at version 3, but keep it Yunnan in the game, then I remove Upper Burma at version 6, but I readd it at version 11, but then take out again at version 13, and then at version 15 I change Yunnan to Upper Burma in the xml, with the consequence that the version 15 will be compatible with itself and 3, 4, 5, 11, 12, but not compatible with 1, 2, 6, 7, 8, 9, 10, 13, 14), and them make changes that don't break compatibility with some previous versions, while breaking compatibility to some other versions past that, or even mixed up. Moreover, having a version number for the map and one for the xml would be very confusing for the users, as they would likely think they are the same (like, they would ask why the hell WAW is saying 3.1 in download and 2.1 in game). For all these reasons, my suggestion to the developers would be just having the version being equal to the number of releases made in that repository, and it being purely a natural number (no decimals), and maybe update the current manual system accordingly, till that happens (for example, world_at_war should currently be at version 14, as said).

Side note, why does the releases usually, but not always, jump a number. Like in world_at_war, why did it go from 0.25 to 0.27, instead of 0.26? Why do we have it at 0.27 instead of 0.14?

Finally, in order to minimise confusion between game's version (in the xml) and release version, I advice you changing the current "v1.0" you see when downloading to something like "Release Number 1". As now, everyone would tend to think that "v1.0", or whatever, is the version of the game (which is not; it is also advisable not using a "vX" notation at all, ever, because that is the one for the rulesets, from v1 to v6, until now).

Thanks for clarifying, and I would suggest you or @ron-murhammer close this and, if you want, summarise the items you are willingly to revaluate in one or more other issues (I definitively suggest you change the current "vX" notation to something different).

A missing element is that the numbering from releases is potentially going to go away so we do not have to set up travis for each map repo. Brand new repo's coming in today will have nothing published to releases since travis is not being setup at the moment.

The system had a version number that was used for download, and as mentioned it was reset ot '1', and my desire is for it to simply be incremented with natural numbers. Any type of encoding there is not enforced, and potentially confusing since we have other similar looking version numbers.

@Cernelius "close pending" just means that this issue is likely to be winding down. It's an ephemeral label and can be removed when it becomes clear there is more to do. Perhaps in other words it's better read as "check that there is not more to do or talk about"

triplea-game / triplea

Obsolete Url And Version Legacy In TripleA Maps Yaml And Versioning Practices #1236

Sun Sep 18 09:36:15 CEST 2016