Handwrite DOM API - Githubissues

Unfortunately, all the DOM's I've generated thus far (via https://github.com/matthewmueller/joy/blob/master/internal/_dom/main.go) all kind of suck. That's not to say it's impossible to generate a good DOM, just quite difficult.

The problem is that they're way too verbose and the files are huge. I think the solution is to handwrite the DOM, copy and pasting most of the logic from the generated output and just making modifications to improve the API and clean stuff up. For what should be included in this DOM, you can refer to this: https://developer.mozilla.org/en-US/docs/Web/Reference/API

I'd like to have as little cognitive overhead as possible going from the JS DOM API to the Go DOM API, so hopefully we can make it look like this: document.Body.QuerySelector(".test")

You'll also have to be aware of this Go bug: https://github.com/golang/go/issues/22866, so for the circular interfaces, it's a good idea to expand all the interfaces manually until this is fixed.

I took a look at the generated code and I agree that they are a bit verbose and the amount of files is quite a lot.

But I was wondering what you were thinking of what modifications that should be done. Like could you give us some example of the generated code and what you would like to be changed?

Also I was looking at MDN docs that you linked and compared it to the generated DOM codes. The generated DOM codes is the whole WEB API and not just the DOM API, so do you want to support just the DOM API that is listed here: https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model Or should we support the whole WEB API listed here: https://developer.mozilla.org/en-US/docs/Web/API

It would be nice to support the whole web api, but then we'll end up with a lot of files, what we could do is maybe split the API's into their own sections like they did https://developer.mozilla.org/en-US/docs/Web/Reference/API on that page so like /internal/dom /internal/device /internal/com or something similar. Would allow better maintence as we wouldn't have a lot of files in one folder, would still end up with the same amount of files but it would be better organized.

But I was wondering what you were thinking of what modifications that should be done. Like could you give us some example of the generated code and what you would like to be changed?

I was thinking closer to how the DOM API is. So instead of:

w := window.New()
w.AddEventListener("click", func(e window.MouseEvent) { ... }, false)

It'd be more like this:

window.AddEventListener("click", func(e window.MouseEvent) { ... }, false)

Also, I don't really like all the htmlbodyelement.HTMLBodyElement, probably better is just dom.HTMLBodyElement or html.HTMLBodyElement or element.HTMLBodyElement. TBH, I'm not even sure most of those are really needed. Maybe for certain ones like tags that have extra properties, but I think most are the same.

The generated DOM codes is the whole WEB API and not just the DOM API, so do you want to support just the DOM API that is listed here: https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model Or should we support the whole WEB API listed here: https://developer.mozilla.org/en-US/docs/Web/API

You're right, probably the whole web API eventually, but DOM would be a great start. I can see us wanting at least XHR, LocalStorage and Cookies, websockets to do any real applications, but those can come after.

It would be nice to support the whole web api, but then we'll end up with a lot of files, what we could do is maybe split the API's into their own sections like they did https://developer.mozilla.org/en-US/docs/Web/Reference/API on that page so like /internal/dom /internal/device /internal/com or something similar.

I agree about this. Just for some backstory, I originally started with one big dom package, but that was really huge and took forever to load. Then I tried breaking it up but got crazy cyclical dependencies. What it's doing now is programmatically grouping the graph by cycles then writing files. This actually does work and there's value in knowing how what should be packaged together but I think it'd probably be nicer if it was packaged by hand based on MDN or something.

One more thing to keep in mind is this issue: https://github.com/golang/go/issues/22866 which was causing some issues with what I generated.

Alright sounds good, I think I understand what you want, I'll do the basic DOM API and then we can see if we maybe should organize the structure more or if it's good.

Once we have done the DOM API and we have found a structure that feels organized and still follows the JS pattern as closely as possible like window.AddEventListener("click", func(e window.MouseEvent) { ... }, false) and such we can start adding one API at a time.

We should also probably do a list of what API's has been implemented or not.

Also a question regarding depecrated methods/properties, should we care that some properties and methods are depecrated in the DOM API or should we just let the browser tell them about it? A good example is the Attr object, https://developer.mozilla.org/en-US/docs/Web/API/Attr It has a huge list of depecrated properties and methods.

Most of the depecrated properties will simply return NULL or do nothing, but it could be nice to get warnings about it when compiling? Maybe an interesting feature would be to add it to the js tag for example we could do something like

// Attributes prop
// js:"attributes,depecrated"
func (*Attr) Attributes() (attributes *NamedNodeMap) {
    macro.Rewrite("$_.attributes")
    return attributes
}

Or something similiar, only problem would be keeping up with which methods and properties are depecrated and such but could be interesting to look into. For now I'll just implement them they won't really do any damage since they will just not return anything special in the JS.

Alright I just wanted to give a update on how things are going and a few questions/things that have come up.

In case you want to take a look then you can take a look at: https://github.com/tryy3/joy/tree/dommanual/dom

I still have some stuff to do before I want some final input on how things look before I do a pull request.

What I have done

[x] Generated the dom code using your existing generator
[x] Put all of the files into more proper packages based on what API they belong to

I generated the code with your dom generator and went through all the files one by one and tried to categorize them into different packages based on what API they belong to.

I initially was just planning to do the DOM API but I noticed that for example the window and document package requires other API's so I ended up simply deciding to implement everything.

Right now they are just categorized and there were several files that was hard to decide on, so I created the utils package. "general" might be a better word as it's basically interfaces/structs that is being used in several packages.

Things that need to be done

[ ] Comment all the files
[ ] Fix package names/importing correct packages
[ ] Finalize the package structure
[ ] Merge some files

The package structure isn't finalized I am sure I will find several files that will be moved when I go through them more closely and if you have any input on the package structure it might change some more.

There are several files that I have noticed that could be merged, some files like https://github.com/tryy3/joy/blob/dommanual/dom/audio/audiocontextstate.go can probably go into a more general file like audio.go in this case. There are also several "init" files such as "eventinit" that can be merged. This is to lower the amount of files.

Questions Do we really need window to be a struct with a New() constructor, isn't window a global instance in javascript and such? I am sure there might be more structs that is similar to window but right now I am asking about window, if we remove the constructor they can simply do "window.Document()" and such to make it look more like javascript.

I am not sure about SVG files and the HTML files. With the SVG files we could maybe put them in their own package? And for the HTML do we really need all of the HTML files (this might require some testing)?

What do you think of the structure so far, there is still a lot to do but am I going the correct direction?

Some notes Regarding that last question, I like the idea that each package should represent an existing Web API instead of just some random package but I was also thinking that in the future maybe we will need to move a few files to make it more similiar to the javascript API. But for a start this aproach will look quite good.

I also had the idea that once we are done with the structure and you like the structure, we could modify the generator to follow the structure and add comments and such based on the manual input. So in the future when new API's gets added/removed we can simply generate instead of going through all the changes in the DOM.

Do we really need window to be a struct with a New() constructor, isn't window a global instance in javascript and such?

No, I don't really like the window.New() at all. If possible, I think it should just be window.AddEventListener(). If the compiler doesn't support that kind of stuff, I'll update it so it does :-D

I am not sure about SVG files and the HTML files. With the SVG files we could maybe put them in their own package?

+1 that sounds good to me – I like the idea of having html.BodyElement, etc. svg could be it's own package too. The "dom" package should probably be broken into "window", "localstorage", etc.

The goal is to have it be familiar (e.g. "I know the DOM API already, I don't want to hunt around for the corresponding Go method") and go-friendly (e.g. "this is how I'd normally write this in Go")

What do you think of the structure so far, there is still a lot to do but am I going the correct direction?

https://github.com/tryy3/joy/tree/dommanual/dom

Looks good!, I think if we can base it off of MDN's categories that'd be good: https://developer.mozilla.org/en-US/docs/Web/API.

Update: https://developer.mozilla.org/en-US/docs/Web is a better example for how to categorize these APIs

I also had the idea that once we are done with the structure and you like the structure, we could modify the generator to follow the structure and add comments and such based on the manual input.

+1 that'd be great. I think if we can automate this as much as possible that would save a lot of time. In the meantime though, I'm happy to maintain a manually written version.

Sorry for the long comment... I don't know how I keep writing these long comments :/ A new update since it's been a few days and I have tried a few different approaches.

A bit of backstory / What I have done So initially I was hoping that everything would go smooth and I was gonna be able to simply go with the web API structure like I explained in earlier comments.

Sadly I ran into several circular import errors, the web API basically inherits everything, so for Go it's a nightmare.

So after I had spent a few days structuring everything without actually testing stuff, I ran into those import errors. So I decided to try and work out the errors, so I ended up making a few scripts that tried to move files around to fix the errors, but sadly I couldn't get anything to work properly.

Instead I decided to work on the generator and work towards generating the structure that I had previously done by hand. Basically what I ended up doing was creating a map[string][]string structure of all the files that I had previously moved around. After that I modified the generator so after it had generated a structure in memory of all the stuff I sorted it based on the map structure.

Once I had that working I made a few changes such as converting map[string][]string into a proper package for future stuff, I will talk more about this later. You can see the package here: https://github.com/tryy3/joy/tree/dommanual/internal/dom/generatestructure

After working on the generator for a few days I managed to get it to work as expected but sadly I didn't get close to my original plan.

Current problem So the generator is working as expected where I can sort the files as close as possible to the original structure that I had.

But there are still a lot of files in packages they don't belong in, in my opinion and I can't move them to more proper packages because they wouldn't make sense or due to import errors.

In the file https://github.com/tryy3/joy/blob/dommanual/internal/dom/generatestructure/structure.json you can look at the "Confirmed" boolean to see if the file can be moved or not. Basically the Confirmed boolean checks if it can be moved without causing import errors and there are a lot of files with Confirmed: false.

While we could try and move files around or create different packages and such I don't think we'll get close to the original idea that I had.

So what's next? Right now the DOM is working but I personally don't feel I like the current structure.

So last time I gave an update I talked to @matthewmueller on Slack and we threw a few ideas around.

In my opinion the best and the easiest approach is to simply put every file in one package called "webapi". If we did that would end up with syntaxes like webapi.Window.AddEventListener which has its ups and downs.

It wouldn't be perfect js like syntax and it wouldn't really be Go idiomatic but in my opinion it's a good middle ground.

If we use a generalised word like "webapi" or something similar it would make more sense than if we used "dom". Because then we can have interfaces and structs like crypto, request, storage etc. and it still would somewhat make sense.

There is one issue that I could see happening and that is there would be a lot of files in one package and that's where my generatestructure package comes in.

Generatestructure package I know it's a dumb name but I was tired hehe...

So the idea of the package is that this is where we'll make our own changes to the files, so right now I have only implemented the "Confirmed" boolean. But in the future I am planning to expand the structure to support things like comments, merges (where we can merge a few files into a single file) and maybe something else that we need.

So the solution to the issue that I could see happening is that we basically generate all of the code instead of maintaining it manually. So if we need to update some comment or similar we simply update the json file (either by editing the actual file and then running some script that goes through all code and export comments and such, or by editing the json file).

We would still have an issue with Godocs but I believe this is the best and easiest way to go, unless someone else wants to jump down the rabbit hole :)

matthewmueller / joy

Handwrite DOM API #81