Json Marker Packs

[coderedart]

Xml Marker Packs

xml_files: xml files are placed anywhere in the pack folder and they will all need to be parsed to get a complete pack. the format is roughly like this: OverlayData is the root tag in xml file optional MarkerCategory tag which is recursive and represents the category menu along with marker attributes which are supposed to serve as "inheritance" properties.

optional POIs tag which contains POI and Trail tags

optional POI tag which represents a marker. two attributes need special mentioning here.

first is the "type" attribute which is an xpath (representing the path to category this marker belongs to).

second is the iconFile attribute which is a relative path from the pack root to a png image.

optional Trail tag which represents a trail. it also has the "type" attribute and its image tag is called "texture".

also has a special attribute called trailData to refer to the path of trail binary data just like png image.

Advantages of using xml packs:

1. compatibility with other Overlays.
2. can add support for attributes slowly one step at a time.

Disadvantages of using xml packs:

1. rust has no mature xml library .
2. editing a marker pack is hard as moving a category menu affects all the markers that refer to them via xpath.
3. the packs in particular have so many errors that blish had to give up and make their own tmllib to have manual parsing with workarounds as they find errors. custom parser means we just lost the whole point of using a popular known format.
4. No defined structure of files. has no reproducibility. once we parse the pack, we will have to remember way too much data like where the marker category tree is in which file, which markers are in which file, what is the order of them etc.. to be able to completely reproduce a pack properly. same goes for order of attributes within the xml tags.
5. case insensitive (atleast in xpath or image/trail binaries paths) so extra steps to normalize case
6. uses UUIDs which are encoded in base64 and no overlay actually ERRORS when there's a duplicate UUID. there's no UNIQUE in the UUID here.
7. having Markers and Trails in the SAME POIs tag, especially when they are interleaved makes deserializing much harder by default in quick xml. having them both in separate tags would have been much better.
8. No separation of markers which are just static and which are dynamic (like having behavior attributes or info attributes which need special care like activation times, text rendering etc..). we will have to check for all of those attributes separately to check if something needs to be done.

In the above issues, some of them are due to lack of documentation/specification of the pack format, some of them due to manual editing of xml files and not checking for errors using linters, others due to no easy editors (which means manual editing) leading to inconsistencies between packs and more or less anyone actually enforcing the pack makers to care about these issues. if a trail binary is missing, there's no error.. so nobody ever knows about it unless they check for it specifically.

Most of the above issues in Xml Marker packs can be condensed into three categories:

1. the lack of a standard file structure + sorting of attributes + sorting of tags like POI or Trail affecting reproducibility.
2. Uses Inheritance of attributes which makes editing harder, as well as dealing with marker packs complicated.
3. Lack of a single source of truth. to look at things like whether color attribute is pre-multiplied or straight, we need to look at the implementation of taco/blish, and if they differ, RIP.

the main issue is that these problems require existing overlays to also support a stricter version of marker packs with the above issues fixed. I doubt they want to do that and break workflows for existing users. So, it is better for us to go with json packs.

I have spent way way way way way too much time on marker packs, and kinda starting to regret this. one of the main reasons to make Jokolay instead of porting taco was that I didn't want to learn specific functionality like taco's UI toolkit which won't be useful to me in the future compared to learning something like Qt or imgui in c++ or egui in rust. Yet, I spent way too much time wasting on this custom format. if it was gltf or some other open format, it might have been okay. we still learnt a bunch of stuff regarding all kinds of formats (text and binary) or the issues regarding representation of data like premultiplied colors or referring to external files like images/trail binaries. but, not enough to justify the amount of time invested.

Why JSON

1. it has support EVERYWHERE. lots of langauges support it natively or via popular defacto standard libraries.
2. has a LOT of tooling/features like jsonpath, json schema, json diff-patch and other RFCs. can be stored in a lot of databases too smh..
3. lots of rust libs like reqwest support it, json-rpc is a thing which is used by vscode for extensions, so obviously supported well enough.
4. easier to edit, format, validate etc.. in all popular editors like vscode or atom. e
5. compared to binary formats, its not bound by specs like protobuf and easily editable by editors.

size difference between xml and json packs was roughly 2 MB for Tekkit's pack.

Databases

just for completion, one more option we considered was databases like sqlite. Advantages:

1. Very Mature and stable. can be used over network if being shared by people
2. can be edited live by multiple people with atomic transactions. (or multi boxers). can be used by multiple overlays too at once.
3. has god tier tooling for stuff like rollbacks, backups etc..
4. overlays like Burrito/Blish/Electaco which have a spike in ram usage when they parse a marker pack, they can now just connect to database and query only the required markers and cache them. no more splitting marker pack into map_named files like blish does to optimize.
5. literally anyone an just make an overlay without having to worry about parsing or caring about marker formats. universal indeed.

Disadvantages:

1. people need to learn sql now, but most can use abstract ORM libraries to ease things out.
2. migrations can be a bit of a headache when changing the marker format schema
3. no git like data version control.

Result

honestly, databases are an attractive option (especially with sqlite). but I don't want to completely commit myself to a schema yet, especially without consulting with other overlays. so i will choose json based pack. eventually we can use sqlite internally, but when we want to export a pack, we can choose to do so in json. once we get enough experience, we can reconsider databases.

[coderedart]

Goals

1. unambiguous. specify everything properly so that we can reproduce a marker pack after parsing it.
2. documentation. clearly mention any type info like sized and signed types of numbers or such.
3. simple. any new overlay should be able to read the docs for a few minutes and be able to understand what all the properties/attribuets are for.
4. testing/validation. we should have examples, along with tools to validate in CI workflows for pack releases, and schema to help such operations.

JSON Marker Pack File Structure

required markers : folder that contains {mapID}.json files which contain markers. required trails: folder that contains {mapID}.json files which contain trails required images: folder that contains all the png files required tbins: folder that contains all the tbin files optional pack.json: file which contains info about a pack like version or name or download link or github repo etc..

optional : all of these CAN be packed into a zip file.

Types used

described in terms of rust structs. the Option<T> means they are optional the Vec<T> vector/array. could be empty. the Map<T, U> maps must have sorted by their key T, so use an ordered_map (cpp) or BTreeMap (rust) atleast when you are serializing to keep the reproducibility. could be empty the Set<T> is an ordered set of T (so no duplicates), and could be empty. All T types will be validated while deserializing, so we will NOT be accepting a marker pack with invalid data.

Marker ID, Trail ID, Category ID, Map ID are all u16 (shorts) numbers. none of them have the potential to reach more than 65k in count. the number of total markers in tekkit pack are barely 35k+ after all these years. Category ID must be unique per pack. Marker and Trail ID must be unique per Map. Image and Trail names must be unique per pack.

JSON PACK

pub struct JsonPack {
    pub pack_description: PackDescription, // from pack.json
    pub cattree: Vec<Cat> // menu. more than one top level category is supported
    pub markers: Map<MapID, Map<MarkerID, Marker>>,// Markers referred to by their ID. 
    pub trails: Map<MapID, Map<TBinID, Trail>>,// Trails just like above
}

Pack Description

xml: doesn't exist

pub struct PackDescription {
    pub name: String, // name of the pack
    pub url: Option<Url>, // url of the pack website if it exists
    pub git: Option<Url>, // if the pack has a git repo, its url can here.
    pub version: Option<String> // semantic version 
}

Cat Tree

xml: represents the menu of categories by referring to them by their ID.

pub struct Cat {
    pub name: String, // name of category matching the xml attribute
    pub display_name: Option<String>, // shown to users in menu. same xml attribute
    pub is_separator: Option<bool>, // whether it just exists as a heading. same xml attribute,
    pub description: Option<String>
    pub children: Vec<Cat>, // its children/submenus. 
}

just recurse through the tree, and use the categories map to get the display name and show it. we will leave it to the overlay on how they want to store their category menu item is checked or not.

Trail

xml: Trail Tag

pub struct Trail {
    pub tbin: Option<String>, // similar to xml attribute TrailData which refers to path. but we just use the name of the file without extension.

    pub image: Option<String>, // the texture name to use for trail mesh, without png extension

}

Marker

xml: POI Tag

pub struct Marker {
    pub position: [f32; 3], // x,y,z coordinates in inches (in game units)
    pub alpha: Option<u8>, // alpha 0-255
    pub color: Option<[u8; 4]>, // rgba bytes as array
    pub fade_range: Option<[f32; 2]>, // fade distance ranges near to far in meters
    pub flags: Option<MarkerFlags>, // a bunch of bools grouped up into bitflags to save space :)
    // some filters like race, mount, festivals, achievements etc..
    pub image: Option<String>, // texture, this also determines the quad dimensions
    pub map_display_size: Option<u16>, // size of icon on map
    pub map_fade_out_scale_level: Option<f32>,// after how many zoom levels will marker fade on map??? 
    pub max_size: Option<u16>,// pixels
    pub min_size: Option<u16>,// pixels
    pub scale: Option<f32>,// scale by this factor

}

MarkerFlags

xml: combines POI attributes ingame, map and minimap visibility, screen edge herding, map zoom scaling, auto trigger and count down as bitflags.

bitflags::bitflags! {
    pub struct MarkerFlags: u8 {
        const IN_GAME_VISIBILITY = 0b00000001; // appears in game
        const MAP_VISIBILITY = 0b00000010; // appears on map
        const MINI_MAP_VISIBILITY = 0b00000100; // appears on minimap
        const MINI_MAP_EDGE_HERD = 0b00001000; // gets stuck on minimap edges like personal waypoint when it is out of view
        const MAP_ZOOM_SCALE = 0b00010000; // scale with map
        const AUTO_TRIGGER = 0b00100000; // should trigger within range
        const COUNT_DOWN = 0b01000000; // show countdown to wakeup time of marker
    }
}

Dynamic Attributes

pub struct Dynamic {
    pub trigger: Option<Trigger>, // if it has any triggers for behavior
    pub achievement: Option<Achievement>, // visibility based on achievement status
    pub info: Option<Info>, 
}

pub struct Trigger {
    pub range: f32, // range in meters
    pub behavior: Option<Behavior>, 
    pub toggle_cat: Option<CategoryID>, // which cat should be toggled on trigger
}

pub struct Achievement {
    pub id: u32,
    pub bit: Option<u32>,
}

pub struct Info {
    pub text: String, // text that should be displayed when within range
    pub range: f32,
}

pub enum Behavior {
    AlwaysVisible,
    ReappearOnMapChange,
    ReappearOnDailyReset,
    OnlyVisibleBeforeActivation,
    ReappearAfterTimer {
        reset_length: u32, // in seconds
    },
    ReappearOnMapReset {
        map_cycle_length: u32,             // length of a map cycle in seconds
        map_cycle_offset_after_reset: u32, // how many seconds after daily reset does the new map cycle start in seconds
    },
    OncePerInstance,
    DailyPerChar,
    OncePerInstancePerChar,
    WvWObjective,
}

[coderedart]

extras: there's also a bunch of other QoL things i did for the sake of jokolay. like making alpha a u8 instead of float, because color attribute is [u8; 4], where people set alpha from 0 to 255, but having alpha attribute be f32 0.0 -1.0 does not make sense. I compressed all the bools in marker pack into a single u8 with bitflags to save space both in json and ram.

instead of trail binary, image paths being hardcoded, i instead made them referenced by a id (u16). and instead of a separate file with image descriptions where we have a map with id as key and {name of image, width , height, source: optional}. same goes for trail files. then we store the image as "1.png", "2.png" etc.. in the images folder. f32 reasons for this:

1. having a string allocated on heap for each marker is just ridiculous. 35k strings.. a u16 is better.
2. images can have their source changed. like "1.png" might not be on filesystem and we can have optional source be a http link. no point in distributing marker packs with images now. download/cache the image. if multiple packs refer to the same image, we can retrieve from cache. and markers won't know anything about this.
3. no more spaces in image paths. one of the errors i found in tekkti's pack was image attribute referring to an image name with a difference case (uppercase in attribute, lower case in actual image filename).
4. i am freaking out that people can put dangerous paths like "../../home/root/waypoint.png". it won't lead to any security issues "yet", but why even allow this.
5. its super easy to iterate over all image descriptions and check that those images actually exist on filesystem. its also easy to iterate over all markers and check that they exist in image descriptions map. just a u16 == u16.

the final and most important change is category tree. categories are a recursive structure in xml packs and they are referred to by the XPath extension of xml. eg: parent.child1.subchild1 (lets call this the "fullname" of a category as referenced by markers") again. huge string allocations in xml packs, and recursive trees are not good for traversal in terms of performance. so, i separated the categories and the tree. there's a cat descriptions map which has a Category ID with {category name, display name, is seperator, Authors {name, email, in game name} }. then, there's a recursive tree structure CatTree { cat_id, Children: Vector<CatTree>}. This is absolutely the best decision i have made.

1. now, categories are referenced by markers with u16 cat id.
2. easier to access any random category instead of traversing a tree.
3. only the display menu needs to deal with tree.
4. there's another overlay made with electron which completely ignores trees and just tells users to search for the pack they need and enable just that. https://github.com/electaco/overlay/releases . this format is flexible enough to work for them too.
5. if you change the category menu, it doesn't affect the markers themselves. example:

in xml:

<MarkerCategory name="fractal_underground" .../>
<MarkerCategory name="fractal_volcanic" .../>
`<POI type="fractal_underground">

if we change the category menu:

<MarkerCategory name="fractals" ...>
  <MarkerCategory name="fractal_underground" .../>
  <MarkerCategory name="fractal_volcanic" .../>
</MarkerCategory>

in json:

CatTree: [{catid: 1, ..}, {catid: 2, ..}]
Cats: {1: {name: "fractal_underground", ...}, 2: {name: "fractal_volanic", ...}}
markers: {34: {... cat: 1, ...}, ...}

after changing menu,

CatTree: [{catid: 3, children: [{catid: 1, ..}, {catid: 2, ..}] }]
Cats: {1: {name: "fractal_underground", ...}, 2: {name: "fractal_volanic", ...}, 3: {name: "fractals", ...}}
markers: {34: {... cat: 1, ...}, ...} // no changes for markers

i wanted unique id for cats, markers, images, trails, packs because of a ridiculous idea about storing markerpacks in a database . allows marker packs to be hosted on servers, allows markers/packs to be edited live by multiple players, allows backups, rollbacks with ALL the billions of dollars worth research/tooling support like gui, cross platform/language support by every language (godot has gdsqlite and other addons), allows multi boxing on local systems where if one gw2 window creates a new marker, other windows will see it too (advanced marker pack creators might use it to try to look at a marker from different perspectives/distances), improved memory usage as we don't need to keep all the markerpacks in ram anymore :) just query database by categories or maps on map change or enabling/disabling categories etc.. i could go on forever.

[coderedart]

Tekkit's pack 3.0 has embraced a lot of blish-hud attributes. source: https://blishhud.com/docs/markers/development/attributes there's a bunch of attributes which are exclusive to blish.

Trails	IsWall	When enabled, a trail will be rendered on its edge like a boundary.
Markers, Trails	Festival	Hides the marker or trail if one of the specified festivals is not active.
Markers, Trails	Mount	Hides the marker if the player is not currently riding one of the specified mounts.
Markers, Trails	Profession	Hides the marker or trail if the player is not currently one of the specified professions.
Markers, Trails	Race	Hides the marker or trail if the player is not currently one of the specified races.
Markers, Trails	Specialization	Hides the marker or trail if the player is not currently one of the specified specializations.
Markers, Trails	MapType	Hides the marker or trail if the current map is not one of the specified map types.
Markers, Trails	Cull	Allows you to make markers or trails hidden depending on which direction they are looked at.
Markers, Trails	CanFade	Allows a marker or trail to ignore fade features such as those which hide trails around the player or between the user and their camera.
Markers	InvertBehavior	If enabled, the behavior attribute's effect is inverted.
Markers	Rotate	Allows you to statically rotate a marker instead of it automatically facing the player.
Markers	Bounce	Allows a marker to "bounce" when triggered.
Markers	Copy	Allows you to set a value to the users clipboard.
Markers	Show / Hide	Allows showing or hiding a specified category when triggered.
Markers	Toggle	Toggles a category on or off when triggered. Similar in behavior to the Show & Hide attributes.
Markers, Categories	Tip	Allows you to display a tooltip on markers on the minimap or categories in the category dropdown.
Categories	DefaultToggle	Indicates if the category should be enabled by default.

[coderedart]

easy to integrate:

defaultToggle => only when parsing the pack for the first time or at resetting the category checkboxes.

festival, specialization, maptype, profession, race => are basic filters to be executed at map/char change events

mount is a dynamic filter based on mumble link

show/hide, toggle, Copy => normal triggers like behavior (we can integrate this into behavior enum)

cull => we will have to generate the mesh/quad in different order based on ccw or anticcw. but does it also exist for 2d markers? in trails we would probably go with up direction as the facing direction.

rotate => 3d direction

painful to integrate:

bounce: very very niche and again needs special attention. (ALL markers need to now have an offset based on some sin(time) uniform added to their y coordinate.) we ignore this attribute for now.

Tip

Allows you to display a tooltip on markers on the minimap or categories in the category dropdown.

tip-name arbitrary text tip-name="Golden Coin" The text to show as the title of the tooltip. tip-description arbitrary text tip-description="Coin 7 - under the cove south of the waypoint." The description text shown under the title.

If tip-name is not provided, the displayname of the parent category is used instead.

If you would like no tooltip to be shown for a marker, set tip-name="". Note that this will also hide the distance indicator and any set tip-description will also be ignored.

Effective Equality

jokolay tries to match the render behavior of taco. we do this by just inheriting the marker's attributes from categories as usual. This is possible because the attributes until now affected either Category or Marker. example:

<MarkerCategory name="hello" displayName="Hello Sempai" alpha="0.5" />
<POIs>
<POI type="hello"  />
</POIs>

name and displayName are exclusive to the Category itself. they are not inherited by child markers or categories. alpha is a marker/trail attribute and will be inherited by all the child markers of itself as well as the child categories recursively to their child markers/categories. so, this is effectively equal to

<MarkerCategory name="hello" displayName="Hello Sempai"  />
<POIs>
<POI type="hello" alpha="0.5" />
</POIs>

from this we get the json pack like this

{
"cats" : { "2" : { "name" : "hello", "displayName" : "hello Sempai" } }, // cats only have cat related attributes
"markers" : { "5" : { "alpha" : 0.5, "cat" : 2 } } // markers only have marker related attributes. no inheritance.
}

we can simply convert this back and it would still be effectively the same as the inherited xml above. so, this is effectively the same marker pack as the original as far as taco is concerned. lets say we add a new marker with no alpha attribute under the same category, this changes nothing.

{
"cats" : { "2" : { "name" : "hello", "displayName" : "hello Sempai" } }, // cats only have cat related attributes
"markers" : { "5" : { "alpha" : 0.5, "cat" : 2 }, "6" : { "cat" : 2 } // markers only have marker related attributes. no inheritance.
}

this is fine, categories don't have any marker/trail attributes when we export to xml, so alpha of marker 6 will be the default as expected (especially when we convert it back to json later). The round trip preserves the effective marker pack. lets replace the alpha with tip-description attribute. example:

<MarkerCategory name="hello" displayName="Hello Sempai" tip-description="whatever" />
<POIs>
<POI type="hello"  />
</POIs>

name and displayName are exclusive to the Category itself. they are not inherited by child markers or categories. but tip-description affects both Category AND Marker.

so, this is effectively equal to

<MarkerCategory name="hello" displayName="Hello Sempai" tip-description="whatever" />
<POIs>
<POI type="hello" alpha="0.5" tip-description="whatever" />
</POIs>

from this we get the json pack like this

{
"cats" : { "2" : { "name" : "hello", "displayName" : "hello Sempai", "tip-description" : "whatever" } }, // cats only have cat related attributes
"markers" : { "5" : { "alpha" : 0.5, "cat" : 2, "tip-description" : "whatever"  } } // markers only have marker related attributes. no inheritance.
}

we can simply convert this back and it would still be effectively the same as the inherited xml above. so, this is effectively the same marker pack as the original as far as blish is concerned. lets say we add a new marker with no tip-description attribute under the same category, this changes nothing.

{
"cats" : { "2" : { "name" : "hello", "displayName" : "hello Sempai", "tip-description" : "whatever"  } }, // cats only have cat related attributes
"markers" : { "5" : { "alpha" : 0.5, "cat" : 2, "tip-description" : "whatever"  }, "6" : { "cat" : 2 } // markers only have marker related attributes. no inheritance.
}

BUT when we change back to xml, we get this:

<MarkerCategory name="hello" displayName="Hello Sempai" tip-description="whatever" />
<POIs>
<POI type="hello"  tip-description="whatever" />
<POI type="hello"  />
</POIs>

Blish now will show tip-description according to their implementation for the category, but more importantly, when we inherit this before converting back to json. we get:

<MarkerCategory name="hello" displayName="Hello Sempai" tip-description="whatever" />
<POIs>
<POI type="hello"  tip-description="whatever" />
<POI type="hello"  tip-description="whatever" />
</POIs>

in json, the new marker didn't have any tip attribute, but now it does because tip-description attribute exists for parent because this attribute affects both marker and category. ofcourse we can explicitly specify that there's no tip-description for the new marker by using an empty value

<POI type="hello"  tip-description="" />

but when we convert back to json, this marker still has a tip-description attribute. although its an empty attribute, it still exists, where as we didn't have an attribute for this marker before. This breaks the reproducibility of a marker during the round trip of xml -> json -> xml -> json. We have to explicitly think of a workaround for this attribute or we lose our reproducibility guarantee.

for now, we will just add use the cat description attribute of json cats. we will output the same attribute for xml packs, so that they don't get inherited by child markers/cats just like displayName or is_separator. otoh, we will just use tip-descriptions for the markers.

[coderedart]

gltf format

1. no support for billboards
2. markers are different from gltf scenes/nodes. they have dynamic attributes like fade or triggers etc..
3. instead of dealing with a complicated marker format, we now deal with gltf AND a complicated marker format anyway because of all the special attributes that markers have

Finally, lets get one thing working properly first. if we want to support more sophisticated features, lets just use nodejs + glfw + threejs i.e wgpu (deno if it gets node modules support). or even bevy / rg3d / rend3.

life is too short and i don't want to spend learning about things like gltf format and implement parsing etc..

having a common format will be helpful for other overlays, but as of right now, there's only blish that has any sort of activity. and trying to fit markers into gltf AND maintain compatibility with taco xml format is too much work for too little gain.

[coderedart]

Burrito has also started to work on making a Protobuf (or other binary serialization) format for Markers.

after looking at gltf, its not "that" bad. babylonJS already uses custom properties in gltf to support things like billboarding. https://doc.babylonjs.com/typedoc/classes/babylon.mesh#billboardmode

[coderedart]

okay, there's no reason for alpha to exist if we already have a color attribute. color attribute can just instead be [0, 0, 0, 0.5] for 0.5 alpha.

[coderedart]

Status Update

1. Json Packs feel too risky knowing that blish / taco could introduce new features to xml packs which could potentially be hard to translate to Json Packs.although, we are close enough that it shouldn't be possible.
2. if we skip json packs and deal with xml packs instead, then, we will need to either force the marker pack into a defined layout like json pack or support some weird per file editing mode. the issue is on how we will differentiate between a defined layout based xml pack vs the current packs when importing.
3. gltf revisited.

I kind of understand how gltf works now and there's a decent enough gltf crate already. to quote myself from https://gitlab.com/coderedart/jokolay/-/issues/19#note_850649157

1. no support for billboards
2. markers are different from gltf scenes/nodes. they have dynamic attributes like fade or triggers etc..
3. instead of dealing with a complicated marker format, we now deal with gltf AND a complicated marker format anyway because of all the special attributes that markers have

but gltf is only a common scene format and rendering is completely dependent on the application itself. so, we can attach custom properties and make sure that the gltf scene fallback works without custom properties for other renderers like threejs, and when you render with jokolay , it will recognize the custom properties of markers.

Advantages:

1. gltf. duh. a common open format. no matter how much time i spend on it, I won't feel bad because i'm still learning something useful.
2. pre-existing renderers. godot (burrito) and threejs (electaco) both already support gltf. so, if we decide to port jokolay to either of those architectures (based on game engine or based on electron or even deno with wgpu), we know we don't have to write a gltf renderer from scratch for basic markers support. rend3 already exists at the moment for jokolay to use in rust to get started with.
3. parsers also exist for most languages.
4. we get free ktx2 support for texture compression. optional ofcourse.
5. if we store marker as a series of 6 vertices that make up two triangles and thus a rectangle. then, we will need to decide on a default size for all markers. this finally made me realize that there's no DEFAULT SIZE for markers documented anywhere. by using gltf, we can easily avoid such inconsistencies between overlays. no more messing around with source code to find such impl details.
6. we don't need 3d model support because gltf is already 3d WITH PBR support inbuilt.

Disadvantages: fortunately, we can transfer most of the design in json packs to gltf. the primary blockers would be

1. exporting gltf back to xml packs.
2. no standard way to support video markers like electaco.
3. size increase.

now we store 6 vertices (vec3) instead of one position (vec3) per marker, but it also means that we don't need rotate attribute anymore. or we could just make markers point primitives instead of triangles. we only need one position vertex. and we can still use rotation property for rotate xml attribute, and the lack of a rotate property means its a billbaord or something like that.

I will try asking Burrito or electaco to see if anyone is interested in a gltf format.

[coderedart]

https://github.com/takahirox/EXT_texture_video for video markers https://github.com/takahirox/EXT_text for 3d text if needed

it seems custom gltf extensions are a good way to actually be able to simply add more functionality to gltf files. https://github.com/takahirox/three-gltf-extensions has a bunch of extra plugins for threejs to support the above unofficial extensions.

just need to be careful that apps which don't support these extensions can easily fallback to core gltf scenes by providing default mesh/materials.

[coderedart]

A list of attributes of marker that are inherited from cats to markers / trails. the only missing attributes are specific to elements like

1. Category = name, display_name,
2. Marker = xpos, ypos, zpos, type
3. Trail = type

Cpu Side custom attributes

Filters:

attributes which decide whether marker is allowed to be active based on data from player.

1. Achievement id + bit. needs api key
2. Festival. needs users to select active festival
3. Mounts. mumble link has active mount
4. Professions. link has it
5. Races. link has it.
6. Specializations. link has it.
7. MapTypes. link has it. ridiculous attribute tho.

Dynamics:

attributes which make marker active or inactive based on custom data that overlay needs to persist across restart

1. Trigger:
   1. trigger_range (distance at which marker triggers if auto_trigger is on)
   2. auto_trigger (bool, whether we should autotrigger when player enters range or wait for him to press F)
   3. Behavior: reset_length and reset_offset. what happens when marker is triggered. sets the next activation time like next day or per day per char etc.. // this is where we need to persist activation times.
   4. invert_behavior: instead of hiding when triggered. display this marker.
2. copy: copy and copy-message which need clipboard and active on trigger
3. Bounce: height, interval and first delay. starts bouncing when being** triggered.
4. tip: show tooltip when hovered over marker. uses mouse position and ray casting instead of distance like other dynamic attributes.
5. info: shows info text when within info-range.
6. toggle_category: show / hide variations. enable / disable the category when triggered.

Xml to Gltf mapping of properties

color -> Color_0

alpha -> Color_0 (just use color mixing to simulate alpha)

height_offset -> vertices. just add to y component

icon_file, texture (trail) -> texture / images

trail_data_file -> mesh bin

iconSize, trailScale, map_display_size -> scale

rotate, rotate_xyz -> rotation

pub in_game_visibility: Option<bool>, // can be custom property. or we can simply create separate scenes for in game 3d, map and minimap. 
pub map_visibility: Option<bool>, // same as above
pub mini_map_visibility: Option<bool>, // same as above
pub anim_speed: Option<f32>, // speed of animation of trail
pub can_fade: Option<bool>, // whether player being under this marker will make it go translucent to not block player character on screen
pub cull: Option<Cull>, // clock / anti clockwise will decide how we order the triangle vertices to cull
pub fade_far: Option<i32>, // fades out completely at this distance, so hidden (but still active for triggers or such)
pub fade_near: Option<i32>, // starts fading at this distance
pub has_countdown: Option<bool>, // shows activation countdown

pub is_wall: Option<bool>, // ?
pub keep_on_map_edge: Option<bool>, // herding on minimap. not possible via gltf.
pub map_fade_out_scale_level: Option<f32>, // ?
pub max_size: Option<u16>, // not possible via gltf
pub min_size: Option<u16>,  // not possible via gltf
pub scale_on_map_with_zoom: Option<bool>,// ?

[coderedart]

and bad news.

gltf is cool for read only packs, but editing them is a sort of messy. the accessors and buffer views and buffers are binary structs and we will never know how to properly edit them.

more importantly, we gltf nodes cannot refer to external gltf files. so, we cannot integrate 3d meshes and edit them easily. it will be highly inefficient and if we use any mesh optimizers, they will remove all the structure of gltf files. its easier for people to export a glb / gltf asset from their blender / editors and put it into a marker pack just like tbin meshes.

its not that we can't use gltf format. its more that we don't gain much by using gltf format if we can't add 3d meshes to a marker pack file properly.but by using json packs, we can eventually add gltf models support to them and get the best of both worlds. gltf files externally edited and marker packs format still easier to edit.

so, we need json packs, and just like Markers / Trails, we can add a new type for gltf objects. it will be similar to Trails using tbin meshes. gltf objects will refer to glb files in a folder. we can easily render the full object / scene with rend3 without needing to worry about enabling / disabling individual nodes inside the gltf.

but we learnt some things like using scale or rotate attributes, so we should continue with that style of marker pack and merge it into json packs. we can also just skip a bunch of attributes that don't interest us for now.

[coderedart]

I think we have come full circle at this point. its time to go over what we want with a marker format.

1. verifiable with schema. will give us clear error when loading / importing a marker pack and validation in CI. we have rschema as well as https://github.com/Stranger6667/jsonschema-rs to do this.
2. diff / patch. although this can be done with treediff crate or serde-diff crate or json diff / patches.
3. easy to edit. really really easy to edit.
4. avoid the messy

we will just keep simple structs for markers and trails.

and use flat fields aid: u32, abit: u32 rather than structs like Achievement {id: u32, bit: u32}.

use rschema to generate json schemas automatically. // hard to deal with bitflag structs. TODO: check that those structs work

to allow multiple people to edit a single pack, we need to have a server facade that takes in changes and sends them to all clients so that their packs actually reflect those changes.

generally speaking, we need to only synchronize changes like creation of a marker / trail / category, where there might be race conditions for taking an ID. editing attributes generally are fine as long as we send the full "edited" property value to make sure that all peers use the latest full value as decided by the main server.

[coderedart]

Final design of the marker packs. i think we need to remind ourselves the current situation.

Advantages of XML packs

1. already designed, so we just need to write the code.
2. reasonably easy to create new markers. XML is reasonably "compact" for this purpose, so usually files appear terse too.
3. backwards compat. we don't need to worry about conversion between json packs <-> xml packs etc..

Why not XML packs?

1. no defined filesystem layout. leading to xml / png / trl files in random paths.
2. no separation of categories. so, category menus are spread across different files. this causes 2 issues:
   1. harder to keep the attributes on a category in sync. eg: a particular category could have different alpha attribute values in different files.
   2. hard to make an editor, as if we add an alpha attribute, do we add it to all files? in which file should we create a new category? how to reparent them etc..
3. Rust xml library support is not great. elementtree is a godsend for now, but it doesn't support formatted output.
4. inheritance of attributes from categories causes unnecessary complexity. hard to edit too, as we need to recalculate all the markers that inherit attribute from the edited category.
5. harder to validate. we need to scan all the files to get category menu, inherit them and then check if all the properties are valid. for example, a marker could have a autotrigger attribute, but the trigger radius attribute could be inherited from parent category, so we need to load everything and process before we can check if the attribute combination makes sense.
6. finally, the ecosystem is already set on wrong footing. we cannot complain about errors in packs, as people will say that the pack works on other overlays, but not on jokolay, so its jokolay issue. we must be as error tolerant (thus rapidxml bindings) as possible.

Why Json Packs?

1. rust serde_json support.
2. json schema with schemars crate.
3. easy validation with schema, but as well as simpler semantics. no attrs are inherited. all categories in one file, so we just need to read cats.json, get images / trls list and start validating each map file immediately.
4. easier to edit live and propagate changes to renderer. because editing a marker or a category etc.. will affect a very specific subset of entities.
5. easier to enable collaborative editing. between operational transform and locking techniques, we can make editing / sharing markers live reasonably smooth. not easy (collaborative editing is never easy), but a smooth experience with reasonably simple implementation goes a long way both in UX and maintenance.
6. defined filesystem layout. easier to reason about where a particular marker exists and diff friendly.

now, we will go through each aspect of JSON marker packs design along with their motivations. we assume everything to be valid Utf-8 encoded. the contents of files as well as the file names.

Files Layout

we separate the marker pack physically into 4 parts.

1. categories in a tree structure.
2. images. all of them in a images directory
3. tbins. all of them in a sub directory
4. mapmarkers. all markers / trails belonging to a certain map grouped into a json file and put under the maps directory

The pack root folder looks like this

cats.json
images/
    {image_name}.png,*many
tbins/
    {tbin_name}.tbin,*many
maps/
    {map_id}.json,*many

Content type of different files

Cats

1. cats.json contains a recursive vector (array) of categories, and each of those might have an array of categories as children.
2. this represents the category menu to be shown.
3. an individual category is referred to by the full path consisting of all its parent nodes. parent.child.subchild.first_subchild and so on.
4. name of the category must be lowercase

Images

1. png files only.
2. the name of the image must be lowercase.

Tbins

1. tbin format only: version: u32 + map_id: u32 + raw binary dump of vec3 (array of 3 floats) structs which represent positions of a series of nodes making up a "path".
2. the name of the tbin must be lowercase

Maps

1. {map_id}.json where map_id is the id of the map according to gw2api
2. each map json consists of markers and trails.
3. markers (and trails) will be arrays and will be sorted by categories so that markers / trails from same category will be together.
4. attributes of a marker and trail as well as their types / docs to be added.

Editing

generally speaking, all 4 components must be edited separately. and editing one should not have an effect on other files. although, mismatch of data can lead to validation errors which must be fixed manually.

Cats

1. categories can basically be considered as a filesystem representation.
2. each category represents a directory which has a single parent directory and possibly multiple child directories.
3. moving categories just like moving around directories.
4. the main edit events are, there's moving up / down existing directories to change their order, editing the attributes of an existing category, creating a category under a certain parent and deleting a category (and its children).

Images

1. create an image
2. delete an image

Tbins

1. Creating a tbin
2. adding a node after a position
3. editing a node at a position
4. deleting a node at a position
5. Deleting a tbin

Collaborative editing

this requires some care to not have race conditions / bugs. we will go over the rough impl details here.

[coderedart]

Conversion rules for Xml to Json:

due to simpler rules, we can just iterate over the entries of a xml marker pack and start creating a json pack.

create a Json Pack with a name (maybe from file name of a marker pack). and start populating the pack as you find a relevant entry.

Validation is done at different levels.

1. Individual validation that an individual entry is okay on its own. markers having invalid properties like vec4 for position. image / tbin references not being lowercase etc..
2. File Level Validation that a sequence / group within a file are okay. checking that markers are sorted by their category in a map. parsing errors of a file. an image being an invalid png.
3. Pack Validation. this is done in relation to other data. checking that the category referenced by a marker actually exists. or image / tbin exists.

Images

1. validate that it is a png.
2. check if an image with this name already exists in the pack with the (lowercase converted) name and add a number to create a unique name.
3. insert the path and its new name into a hashmap for markers / trails to refer to and use the new name when deserializing.

Tbins

1. pretty much the same steps as Images.

XML

1. add the category tree to pack's category tree. make sure to have inherited templates inserted into a hashmap for use by markers / trails.
2. add markers / trails by looking up the image / tbin new names from paths and inheriting attributes from category templates.

[coderedart]

this issue is all over the place. lets make a new one.