Compiler flags (and output?)

[oderwat]

Hi @janpfeifer, I have some more special questions:

• Can we add compiler flags? I was writing some benchmark code and wondered if I can get the output if go build -gcflags="-m" somehow (yeah, crazy to do that in the notebook).
• Could we run the code with go test instead of go run? I guess that will only work for the full notebook, though?

Both if it is not critical. I just wonder if it would be possible and "easy" to implement?

[janpfeifer]

hey @oderwat , notebooks are made for experimenting and demonstrating things, whatever it is. So the ideas are great and make lots of sense!

Just brainstorming on these ideas:

Compiler flags

Would a special command %goflags <flags> where everything inside are passed to the go build command ? This is super easy to implement -- if you agree with the format, I'll implement it in a coffee sitting (most of the time will be spent adding documentation probably).

Running notebook/cell as test:

I can imagine two different ways of working this:

a: %gotest a cell-only command, makes the current cell execute as go test, but doesn't affect other cells. b: %gotest is global, and every cell starts behaving as go test. Then we add %gobin to return to binary format.

Some considerations:

• I was going to suggest that functions TestXXX are only included/executed in the cell they are defined, as opposed to all cells. This is useful for notebooks that mix normal cells with test cells: the normal cells don't need to see the TestXXX functions, right ? Also if one splits TestA and TestB in two different cells, we can run TestA only in cell A and TestB only in cell B, and not re-execute all test functions in every cell. What do you think ? (except to TestMain that likely should be present in every cell execution). Maybe we also add a %gotest all to actually run all tests defined in a cell ?

• GoNB splits the compilation and execution of the code. So it doesn't use go run, but rather go build and then executes the binary. This simplifies error parsing. Do you see any issues if for %gotest we do the same and first go test -c (which builds the test) and then execute the built binary ?

• Now if one selects %gotest, what do we do if in a cell someone defined a func main() (or a %%) in the cell ? Issue an error/warning and refuse to run ?

Implementing this I think would be straight forward (not difficult), but some amount of work: one would need to generate another file (main_test.go) -- which involves having separate file line numbers mapping to cells -- when running tests (but probably not when "auto-completing" or "inspecting" with gopls), and specialize the generation code not to generate func main.

A completely different approach to testing in the notebook would be not to change GoNB, and instead just use it as usual, but call testting.Main to call the Test... functions from a normal binary. It's not well supported based on the docs, but I would expect it to work ... Wanna try it ?

[janpfeifer]

So I was having coffee, and experimenting with %goflags. See here:

https://github.com/janpfeifer/gonb/blob/goflags/examples/goflags.ipynb https://github.com/janpfeifer/gonb/blob/main/examples/tests/goflags.ipynb

And it occurred to me that while setting the flags is easy, GoNB still doesn't display the output of the compilation if it succeeds -- this could be changed though, maybe it makes sense.

But also for the particular case of -gcflags=-m case, probably it can be easily checked by just doing:

!*go build -gcflags=-m

In a cell. Check out the notebook linked above.

Still, %goflags could be useful for other cases as well -- and it's trivial to add (actually it's already implemented in the branch). Any thoughts ?

[janpfeifer]

The %goflags is documented and submitted, it will appear in the next release.

[oderwat]

@janpfeifer !*go build -gcflags=-m was actually what I needed originally. And it works so easy, even when the output is a bit 'unrelated' to the cells, of course. Thanks!

For the other flags I checked the example and that looks excellent.

Regarding the tests, I am not really sure if I have a good idea about how it should work in the big picture. At first glance, I would want to have a test cell and the tests from that cell should output there (with an optional -v). I think they should be a separate package, so they can only test the exported API (by default?).

[janpfeifer]

Nice. For the !*go build ... I'm wondering if I should parse the output and add the notes of which lines those represent in the cells. It could be done.

For the tests, my initial thought was to write the test functions to a main_test.go (pkg main), so it would be the same package, and it would have access to everything. Since this is also the more common case ... But I also don't have a clear view on how this would be used.

When developing (for instance GoMLX) I usually have a code opened in GoLand, and I write scripts and tests in GoNB until I'm happy with what I get, and then copy parts of GoNB code into a _test.go tests. So I can see the benefit of writing the Test*() functions directly in GoNB. Also for writing educational material, blog posts, etc.

Is it a similar use case you are thinking about ?

[oderwat]

My original thought was about writing (micro) benchmarks using the test package.

Besides that, I have a similar idea like you, in writing some code or tests in the notebook, especially as a demonstration for others to check out. Kind of like sending some notebook links around instead of Go Playground (or Gist / Repo and so on).

I am currently trying to get a setup that makes the Notebooks part of a shared development/testing/staging environment. Kind of like an interface info the whole system. We also started to add notebooks instead of "examples" in the repositories. This is all still a bit experimental, though.

[janpfeifer]

It was a bit tricky at parts, but I think I got a first version working, including benchmarks, check it out!

Here the notebook I use for testing.

[oderwat]

This is great @janpfeifer. When I started to experiment with 'gonb' at first, I was not expecting that this will get such a quality tool that quick! Great work!

[janpfeifer]

Thanks @oderwat , really appreciated!! It is a side project, but I take pleasure in trying to make it well done :)

Btw, I cut a new release (v0.8.0) for the new features, since I didn't see any other issues.

cheers

[janpfeifer]

Btw @oderwat , on the same line, I just implemented %wasm, that compiles the cells as a Wasm program, and runs it in the notebook.

This can be handy to play around with Wasm and creating widgets in Go.

Very new and experimental, but in case you have interest in this use case.

[oderwat]

@janpfeifer That is very nice. We are using Go-App in production. I am a big fan of having just one language in projects, and I prefer if that is not JS (or TypeScript).

[janpfeifer]

Same here, one of my projects I'm developing a game using Go+Wasm+WebGL, and it's working super nice, almost no JS needed :)

[janpfeifer]

Btw, Go-App looks super cool! Let me see if can create a demo of Go-App running inside a notebook.

[oderwat]

OK. I have some project here that I made when starting to evaluate our new technology stack, go-nats-app. It is mega cool to use a front-end in Go that actually can go directly by web socket into our NATS Cluster.

[oderwat]

Just as a little teaser: You can create a channel for each running application through NATS. Like having an admin tool that connects into the PWA running on somebodies phone, for example. Without any "extra" stuff. That is just part of the technology.

[janpfeifer]

This is so cool ... I had to watch the presentation about NATS, next time I need to coordinate communication among parts I'm going to use it :) -- when I manage to find time to distribute training in GoMLX I may use this.

And very cool it uses websocket, and it all works in Wasm -- took me a while to get that working correct in my game.

I hadn't used the concept of PWA before. I'm still not sure what in practice it means though -- after reading/searching a bit around.

[oderwat]

It basically means that all data that you get from the server is first cached inside the browser. So if there is no internet, the full application is still there and can run. The browser loads the data from the internal cache. The PWA can get installed as an application on your desktop (using Google Chrome or MS Edge) or mobile device (iOS/Android). It then looks like a native app. But you did not need to go to any "shop". The app itself usually talks using REST or web sockets with one or more backends. So for example: You can create a HTML based app that uses "canvas" for a game and it stores the high-scores into a NATS-cluster. We also started to develop desktop applications with Go-App using webview (which can be packaged as a app) similar to Electron, but with the Go-App based declarative front-end.

[janpfeifer]

The PWA concept is very handy, gotta try it sometime.

And webview: I wish I knew about it when I did hiveGo, it would have saved some time. I should ressurect hiveGo and use that as a demo in GoNB Wasm -- it is a fun game (Hive)

Btw, do you do all this for work, or mostly hobby projects ?

For me all of these are hobby projects -- but I recently had a long vacation, then I got to code more.

[oderwat]

It is a mixture between work and hobby. I am CTO in a small company of a near lifelong friend and can influence our strategies and lead the team. Since I have been programming since 1982 and have numerous different interests, there is always some overlap between work and hobby. So my hobbies more often than not pay out eventually, but I am not directly paid for it.

[janpfeifer]

That's smart! My hobbies and work do interpolate, but more tangentially -- although I can't complaint much, my current work projects are quite interesting (TF-DF, TF-GNN).