search

MakieOrg / Makie.jl

Interactive data visualizations and plotting in Julia
https://docs.makie.org/stable
MIT License
2.37k stars 302 forks source link

VizCon planning #504

Closed mkborregaard closed 1 year ago

mkborregaard commented 4 years ago

Hi, We've got the VizCon coming up a month from now, and we should start to build up the program. There'll be a welcome email coming up with some practical details as well, so this is for the content. I suggest doing it here on github, as discussions here tend to stay a little more on point and hit a crowd a little more focusedly than on discourse. Let's just ignore that this is the AbstractPlotting repo - it's a nice way to keep things together.

cc @kristofferc @tlienart @asinghvi17 @SebastianM-C @piever @logankilpatrick @c42f @daschw @pfitzseb @jkrumbiegel @SimonDanisch @strangeli @jheinen @vilim @oxinabox @glennmoy @BeastyBlacksmith

The idea is that we have 4.5 days together to give a massive push to plotting / visualization in Julia. Many attendees are involved with packages in JuliaPlots but the game is of course open for work on any aspect of plotting in Julia. If a big enough group agree on a single goal - like pushing Makie from "really promising" to "a relevant option as the daily plotting tool of choice for a larger group of people", it should be possible with the right coordination and planning of tasks. So let's start that process.

In practical terms, Thursday to Sunday will have roughly this structure:

On Monday, we will have a half day (ending at lunch) for summarizing and planning follow-up steps.

We'd like to have some presentations-for-contributors of packages, and some of the group work can take the form of discussion groups. Here are a number of projects and presentations we've already talked about having in there:

Projects

Other projects could include building recipes for those packages that don't have them yet, making StatsPlots recipes for models definable in StatsModels, finalizing the pgfplotsx backend to Plots, making VegaLite consume recipes, etc etc.

Discussion groups (perhaps plenary, perhaps in groups)

Presentations-for-contributors

We hope to have a mix of discussions, coding individually, in smaller groups and teams, presentations, and fun!

To ensure progress it would be great to have some themes - it would be great if anyone would feel like leading one of these themes, and being responsible for identifying some issues to work on. These could be

Themes

These are just stubs... now let's fill them in!

So, to everyone here, could you report if you have anything in particular you plan to be working on, suggest one or more projects, consider whether you could give a presentation of your favourite package or project, or lead up a theme? This is the kind of coordination that will make sure we all make the most of this opportunity, which essentially arrives extremely rarely in Julia development! Looking forward to hearing your input.

mkborregaard commented 4 years ago

Also, @kescobo @sjkelly @mcmcgrath13 (am I missing anyone?) have announced that they would like to join remotely. Sorry for forgetting to tag you above - we'll do our best to give you a feeling of participitating though you'll have to do it virtually :-)

davidanthoff commented 4 years ago

I might be able to join remotely for a bit. I’ll be in Munich, so at least time zone wouldn’t be a problem.

KristofferC commented 4 years ago

I can talk about PackageCompiler and some of the design choices we made in PGFPlotsX.jl that I think contributes to us having relatively few issues/bug reports.

kescobo commented 4 years ago

Given the timezone, I'm not certain how much I'll be able to join real time; do we have any further info about A/V and recording? If it's not possible, I'll at least try to catch up on issues and slack, though I may be doing most of my contributing while you're all out having beers...

On that note - can any locals send me a few good local brews so I can look for distributors near me? I figure this is a good excuse to do some drinking at work 🍻

asinghvi17 commented 4 years ago

@mkborregaard, we should probably put down a paper discussion group, even if just for an hour or so.

mkborregaard commented 4 years ago

All great ideas - I really like the idea of participating remotely by drinking the local beers :joy:

Tokazama commented 4 years ago

I'd love to contribute wherever I can throughout this. My schedule is super unpredictable lately though, so I'll just try to follow along and chime in and contribute when possible.

jheinen commented 4 years ago

I will be there on Friday and Saturday. If the schedule allows, I would love to demonstrate the new features of GR and also explain our deployment strategy (CI/CD). This is very important for the integration with other packages (Plots, GRUtils). At the moment I don't know if GR is still relevant for Makie. I have the impression that CairoMakie is quite sufficient. But I am very interested in an integration with current or future recipe systems (AbstractPlotting , etc.).

piever commented 4 years ago

There are a couple of topics that I would like to discuss (not sure about the format).

I think that StatsMakie needs a bit of a "push". I could probably start by introducing the StatsMakie design and API (grammar of graphics in Makie). Then it would be great to clean and finalize the implementation. This is closely linked to the recipes discussion: ideally, it should be possible for the StatsMakie pipeline to just return a list of plots with "recommended attributes", such as layout, or axis labeling.

The second topic is the relationship between Makie and Interact. This in turn has two aspects:

The first topic (StatsMakie) should be feasible within the conference. The second topic is more of a long term plan, but I think it's worth discussing here, especially since right now it is a bit difficult in julia to make apps where both the UI and the plot are interactive, and Interact + WGLMakie could potentially address that.

asinghvi17 commented 4 years ago

@jheinen, at the moment, CairoMakie doesn't support 3D (and has quite a few weird bugs). If we could get an interactive GR backend up, that would be really lovely! It would also be nice to support some form of TeX math in Makie (which I think GR does).

I think someone brought this up before, but it would also be great if we could integrate CoordinateTransforms and GeometryTypes. That would allow (IMO) a much more robust interface for transformation layers going forward.

SimonDanisch commented 4 years ago

I have no personal attachment to cairo, besides that it was easier to get things working for me... If I had some support from @jheinen, that would easily offset that advantage I suppose...

asinghvi17 commented 4 years ago

We have https://github.com/JuliaPlots/GRMakie.jl which used to at least draw lines before the backend API changes. Maybe we could use that as a starting point?

asinghvi17 commented 4 years ago

I've attempted to codify what we know so far about potential projects and talks into the Vizcon 2 project in JuliaPlots. The new columns are Projects and Talks.

Please feel free to add yourselves to them if you want to lead a project, or give a talk about something!

Tokazama commented 4 years ago

@asinghvi17, lots of nice stuff there! The CoordinateTransformations bit looks like a really nice clean one too considering how much it's used other places.

Are there any places where the new plots recipes stuff is being discussed or developed more specifically? I have some ideas but I'm not sure whether they are even applicable because I have now idea what's going on there.

asinghvi17 commented 4 years ago

I think Simon was going to present a working example of Makie consuming RecipesBase's recipes on the fly at Vizcon, and Julius has a prototype of a new recipe system for Makie in MakieLayout.

mcmcgrath13 commented 4 years ago

I'll be with @kescobo on the local beer participation train as the time zones are rough. I'll do my best to keep up with the conversation and follow along.

BeastyBlacksmith commented 4 years ago

I can give an overview of the current status of the pgfplotsx-Plots backend

asinghvi17 commented 4 years ago

@jkrumbiegel would you be able to give a high-level talk on MakieLayout (preferably on the first day) for the integration project?

jkrumbiegel commented 4 years ago

Sure :) But I don't yet know definitively if I can already come on Thursday or only on Friday afternoon.

vilim commented 4 years ago

I will be happy to work on anything, though I have more experience with the Makie than the Plots ecosystem. I will be arriving on Thursday afternoon.

kescobo commented 4 years ago

@mcmcgrath13 First we need someone from Berlin to send recommendations... C'mon people, don't make me do my own research!

BeastyBlacksmith commented 4 years ago

Well there are the big brands "Berliner" and "Berliner Bürgerbräu" and a lot of microbreweries, which I don't know if you can get them abroard like "Quartiermeister", "Vagabund", "Eschenbräu" a minimalistic one called "Bier".

asinghvi17 commented 4 years ago

@mkborregaard, could you post the details about the seminar room TU Berlin has given us over here? They seem to have been deleted from Slack.

I was curious about the possibility of streaming presentations live, similarly to how JuliaCon did it. Would that be possible at all?

oxinabox commented 4 years ago

I was curious about the possibility of streaming presentations live, similarly to how JuliaCon did it. Would that be possible at all?

Live streaming presentations is always either incredibly labour intensive, or incredibly expensive.

Tokazama commented 4 years ago

I would be happy if it was just recorded and action points relevant to the workgroups were made available live somewhere for people trying to participate remotely.

asinghvi17 commented 4 years ago

We could also ask the presenters to capture their screen and audio (from the presenting laptop itself). That seems like a cheap and easy option, and we can always upload to some platform later.

logankilpatrick commented 4 years ago

I’ll try to take care of this ( I have a mic and a camera that I’m bringing) for both recording and remote participants. I will be attempting to film and set up a Plotting/visualization course for Julia Academy which is why I am attending VizCon!

asinghvi17 commented 4 years ago

Perfect, we can figure something out there itself. Would JuliaLang be able to upload the videos onto their Youtube channel as well (for public distribution)? I think they did that for JuMP-dev...

j-fu commented 4 years ago

@mcmcgrath13 First we need someone from Berlin to send recommendations... C'mon people, don't make me do my own research!

The hyperlocal stuff I don't normally drink is Berliner Weisse, see e.g. here and its on tap in MA...

Just sent an email to register, plan to attend Sat-Mon. Working on this one: https://github.com/j-fu/VTKFig.jl meant as a showcase what can be done with vtk.

mkborregaard commented 4 years ago

@strangeli is responsible for the room booking at Technical University - Lia do you know the answer to Anshul's question?

SimonDanisch commented 4 years ago

I started slowly going through issues & managing https://github.com/orgs/JuliaPlots/projects/2! I'll also add a few project descriptions!

strangeli commented 4 years ago

@mkborregaard, could you post the details about the seminar room TU Berlin has given us over here? They seem to have been deleted from Slack.

I was curious about the possibility of streaming presentations live, similarly to how JuliaCon did it. Would that be possible at all?

This question? The room is our seminar room (EN223) at the Control Systems Group, it has a beamer and a blackboard (!) and space for around 30 people. There is no special streaming equipment in place and I honestly wouldn't know what equipment is needed for that. I have one paper deadline on Monday which is quite tough for me... :-/ If anyone sends me a list of technical equipment (camera+stand/microphone/???), I could try to find out next week if I can borrow it somewhere.

My student time is over quite a bit such that I might not be fully up to date but generally the rooms with good equipment for whatever are the large lecture rooms which have these fixed seats that go "uphill" and live-streaming is not a usual case here either. For a technical university, things are still quite analog here or I don't know them, yet :-). I know that they started to film some lectures though so that there must be at least one camera around at that university. I promise, I will at least try to hunt it down. ;-)

Sorry that I don't have a better answer. Looking forward to meeting you anyway.

j-fu commented 4 years ago

I can offer a talk on vtk for Sat or Sun, kind of "Whats in vtk which would be nice to have in Makie, with examples". Essentially its scalar and vector fields on 2D and 3D simplex meshes and it would be based on a Julia wrapper around a C wrapper around a C++ wrapper around the vtk API. May be not exactly what can be done immediately for Makie, but just some ideas for thoughts on directions.

mkborregaard commented 4 years ago

Thanks @strangeli . It sounds like @logankilpatrick is bringing a camera, so you shouldn't need to be hunting around for one. Good luck on the paper deadline!

oxinabox commented 4 years ago

https://en.wikipedia.org/wiki/Unconference

mkborregaard commented 4 years ago

That's an awesome term. VizUnCon!

c42f commented 4 years ago

If a big enough group agree on a single goal - like pushing Makie from "really promising" to "a relevant option as the daily plotting tool of choice for a larger group of people",

Personally I'd love this to be the overall goal, and I'm keen to work on any project which we decide has the best chance of making it happen. Layout and recipes seem to be two key things there.

I also wonder whether there's a way we can move from Plots to Makie via evolution (not revolution, because IMHO that rarely seems to work out very well)? Yes, there's an impedance mismatch, but could we revisit AbstractPlotting as a viable Plots backend?

As a user, I really want to use Makie for all the things, but for everyday use I still find myself going back to other systems for various reasons. It's a bit of a death-by-1000-cuts in needing obscure features more often than you might think, but also in not understanding how to fix things and being confused by the documentation.

One thing I'd like to get from this meeting is a really solid understanding of how the Makie ecosystem is put together at a high level such that I can both use it and hack on it effectively.

With that in mind, here are a few things which have confused me in the past, and subjectively led me to feel like I didn't "get" the API in a fundamental way.

Now, I haven't spent any time looking at these things for perhaps six months or so — maybe things have changed — but a lot of these issues could be addressed with improved documentation. So a possible project I could lead would just be to learn as much as possible from the experts (that is, ask all the dumb questions!), and to attempt to capture that into both improved user docs and devdocs to help other developers get going.

mkborregaard commented 4 years ago

Personally I'd love this to be the overall goal

I agree that this would be a really good overall goal of the (un-)conference. Still - we should maintain that this is open - everyone should work on what they feel like the most, and anyone who will come with a project and inspire a group of people to help should just go ahead with that. I imagine that some people will be leading ideas and projects, whereas others will mainly help. But agreed, if a large proportion of people want to make this happen, I think that would be a really great outcome. My main concern is that Simon is going to have a limited bandwidth - so if many people should focus on Makie it requires that the people who know Makie really well think about how they can stimulate this progress.

could we revisit AbstractPlotting as a viable Plots backend?

I think actually RecipesBase capability of Makie could solve this another way around. Given that (almost) any Plots command can be expressed as a recipe it should be easy to create a link from Plots to Makie once that is in place.

mkborregaard commented 4 years ago

Hi all, I've made a very preliminary program - this is just a suggestion to say something about how the days may look, how we can organize the time, etc. Thoughts? How do we collaborate best about this?

Screenshot 2020-02-22 at 12 09 14

I also updated the list of attendees

jkrumbiegel commented 4 years ago

I want to learn more about the possibilities of the different Makie backends. I have a very good mental model of how a library like matplotlib assembles a plot description and how a backend converts it to pdf or png.

But I think because GPU based plotting is more "stateful" (buffers need to be allocated, shaders loaded, drawing is not as sequential but more parallel) I don't understand what we can or can't do, especially because it gets so complicated when we talk about interactive plots in GLMakie, web based stuff in WGLMakie and static plots in Cairomakie all at the same time.

Not understanding how scenes worked inhibited the initial MakieLayout work quite a bit, for example. I honestly still don't even understand how these nested plot objects / recipes work in Makie, but maybe this system will be changed during vizcon anyway. Right now I just see a forest of conversion functions with unforeseen effects :)

So hopefully after VizCon I'll be able to contribute much more efficiently. My goal is to establish a stronger foundation with scene clipping and axes, for example, that support data transformations so we're not so limited to standard 2d and 3d plots anymore in the Makie ecosystem.

jkrumbiegel commented 4 years ago

To add to this, I think it's best if we concentrate on the low-level building blocks of Makie and don't get too focused on high-level API questions. In my opinion the foundation needs to be really stable and powerful before we can build on top of that. Right now there are still many small issues, for example with buffers for multiple attributes, or text display, or clipping, or composition of multiple plot objects, and so on. These should ideally be solved before writing recipes unless all the recipe work should be redone at a later point when low-level things change..

SimonDanisch commented 4 years ago

To add to this, I think it's best if we concentrate on the low-level building blocks of Makie and don't get too focused on high-level API questions. In my opinion the foundation needs to be really stable and powerful before we can build on top of that. Right now there are still many small issues, for example with buffers for multiple attributes, or text display, or clipping, or composition of multiple plot objects, and so on. These should ideally be solved before writing recipes unless all the recipe work should be redone at a later point when low-level things change..

I'd like to second that...Last vizcon I think we (I) spent too much time on discussions that were pretty irrelevant to the concrete things that need to be done now - this time i'd personally would like to spent more time to get things done... Of course, people who want to discuss bigger, theoretical things, can still do this in a group :)

asinghvi17 commented 4 years ago

Agreed, core development is probably best done in person.

I'd like to give a small (10 min or less) talk about MakieGallery and its architecture, and solicit some suggestions for improvements, if we have the time for that.

mkborregaard commented 4 years ago

@SimonDanisch so do you suggest we remove the "group discussions" and move them to smaller groups? Great idea to focus on internals, but can 21 people do that at the same time? Maybe better to have "internals team" and "interface team" etc?

c42f commented 4 years ago

My goal is to establish a stronger foundation with scene clipping and axes

On a very nuts-and-bolts level, I'd be interested in the various aspects of making clipping work properly including the GPU and shader side of things (assuming that's still an open problem).

asinghvi17 commented 4 years ago

It still is, to the best of my knowledge.

SimonDanisch commented 4 years ago

Well, scenes now rectangularly clip their content, and @jkrumbiegel uses this in MakieLayout to do correct clipping..but he got greedy and now also wants circular clipping etc :D

asinghvi17 commented 4 years ago

Well it's not just him ;) - arbitrary clipping would be needed for earth plots!

strangeli commented 4 years ago

Hi, I am kind of back to that type of life again which does include communication, be easy on me though... I was never good at short texts.

Here is what I felt responsible for so far w.r.t organization:

  1. The secretary will give the attendees list to the door people and I will update it on Friday, Mach 13.

  2. I will not be in Berlin on Thursday, March 12. Simon D will organize the keys for the first day (if you want to open/lock the room etc), from Friday on we can use my key.

  3. If needed, we can use up to 5 rooms on Sat/Sun - that is where my key fits and no one is here anyway. The other days except for Thursday (i.e., Fri/Mon), we could co-use some of those rooms, too.

  4. I ordered 25 guest accounts for internet access in case people have no eduroam.

  5. Our technical assistant is on vacation. I may ask her next Monday if she has equipment for streaming if this is still needed. (I found a Canon 600D so far but it seems not suitable for streaming over long periods of time.) According to what I read here, we do have a cam, a mic, that is enough?!

  6. Restaurants (I don't care, just because I was asked): My favourite around here is currently only open till 6pm (weird...) [1], then there is a very close and funny restaurant on a ship [2] and on my beloved long and lonely evenings in the office I order food or go to [3], [4] by bike and there is a lot of food in Moabit (near Turmstraße) not so far away. Hence, I will give you my customized suggestions on Friday Mar 13 depending on your willingness for mobility and adventure.

  7. And for drinking - this is like a really traditional bar in that area: http://www.zwiebelfisch-berlin.de/, https://goo.gl/maps/vhdnhvaWCdYYqvu57

...? Lia

[1] Used to be a beer garden: http://www.schleusenkrug.de/contact.html [2] Ship: https://www.capt-schillow.de/hier-finden-sie-uns/ [3] Vegan/vegetarian Indian restaurant: http://www.mysatyam.de/anfahrt.html [4] Vietnamese/sushi around Savignyplatz - seems that I can't find it online. It is either closed or it is so cool that it has no google maps. Here is is still as Lemongrass https://www.openstreetmap.org/node/26735763#map=18/52.50736/13.32075 with a non-working website link. Anyway, I might find out. I always go to that corner and there is some Asian restaurant changing names once in a while.

asinghvi17 commented 4 years ago

Thanks for the info, Lia! I had one other question - does the seminar room come with chalk included? I'm bringing along 8 sticks of Hagoromo, was wondering if that would be sufficient (I have had pretty bad experiences with the standard chalk brands, at least at my school 😄)