Better database discovery

[loicknuchel]

Hi,

I'm the creator of Azimutt and just made a way to open any database structure file in one click, allowing anyone to understand it more easily. It can be added with a badge, so here is my contribution in making your project even more approachable for your users.

Feel free to merge if you like ;) Loïc

[HolgerJeromin]

I get a redirect from https://azimutt.app/create?sql=https://raw.githubusercontent.com/openstreetmap/openstreetmap-website/refs/heads/master/db/structure.sql&name=OpenStreetMap to https://azimutt.app/00000000-0000-0000-0000-000000000000/00000000-0000-0000-0000-000000000000 which is not helpful...

It actually works (only the url in the browser is suprising).

Your project has 52 tables. Here are some that could be interesting:

Not sure if this service is worth linking...

[loicknuchel]

Hi @HolgerJeromin Thanks for the quick feedback, indeed there is a redirect once the project is loaded. This is not just this screen but a dynamic app to explore your database. You can click on suggested tables or search on the top left bar to display tables and follow relations. Here is an example diagram I built with:

[loicknuchel]

I added a documentation page to demonstrate it and explain how it works: https://azimutt.app/docs/badge Let me know if I should still improve it.

[gravitystorm]

Thanks @loicknuchel for your suggestion.

I've tried using it, and it looks great. But when I've tried using it used it a couple of times, I haven't found it very useful, and I give up after a minute or two. I think the main issue is that it doesn't load all the tables at the start, so you can end up with just a couple of tables showing (e.g. you click on "acls" from the suggestions list, and then click "users", and then that's kind of it? I can't see how to click to add more). There's a search box, but people who are exploring the db don't know what tables to search for.

I think it would be more useful if all the tables were shown, and nicely laid out, but perhaps that's overwhelming or unclear without manual arrangement and grouping. As far as I understand, that's a paid feature.

I realise that I'm not the target user for this, since I obviously know the overview of the database structure and how everything works. But for the details I wouldn't use it either, because annotate_models puts the information we need into the model files, i.e. closer to where we work.

So my request here is to other contributors, particularly new contributors - can you try it out please? Have you found this useful, and if so, what did you learn? Do you need a whole-database overview when you start working on the project?

[tomhughes]

There's a button or something somewhere you can click to turn on the automatically detected relationships but I admit it confused me as well.

[loicknuchel]

Hi @gravitystorm, Thanks for trying it and for your feedback, very interesting.

Loading all the tables at first is a real issue as it's often overwhelming, even in your case with 52 tables. I also have an issue on my side as the auto-layout algorithm I use is not very good :sweat_smile: And you are right, the free plan is limited to 10 tables displayed on a layout.

The intended usage is more toward long term usage with handcrafted and meaningful layouts than a single one with everything, but I get it doesn't fit well with Open Source project like yours. I will look for a better auto-layout algorithm and maybe allow more displayed tables on the free plan on the first layout.

I also see I have to improve the first interactions, especially I should find a way to make clearer how to navigate following relations (you can click on colored icons, primary keys and foreign keys, to display the related tables and build the useful schema). On the searchbox, I will also add table suggestions so if you don't know what to type, you can at least pick some of the central tables.

Interested by other feedback and eager so see if it could be helpful for you with a few improvements ;)

Cheers!

[loicknuchel]

Hi! Following your feedback I improved the first experience on Azimutt:

• much better auto-layout for showing more tables
• improved the welcome screen to show more tables in one click
• no more table limit on the initial layout
• table suggestions on the search

Let me know what do you think about it and if it makes Azimutt more useful from your perspective. Cheers!

[nenad-vujicic]

So my request here is to other contributors, particularly new contributors - can you try it out please? Have you found this useful, and if so, what did you learn? Do you need a whole-database overview when you start working on the project?

I started learning the OSM source code about six months ago. One of the first things I did was to create a printed visual representation of the database schema to quickly get an overview of the project's structure. Over the next few months, I used this printout to gain a better understanding of various parts of the project and how they are connected (e.g., counters, tag, comments, tags, nodes-ways-relations etc.). So, yes, I believe this utility (and similar ones) can be very useful when starting out.

However, before using it daily, I would like to see:

1. Improved layout management: for example, tables still overlap, the most important table(s) should be placed at the center, ... However, improving these features also increases the complexity of the algorithms.
2. Improved interactivity: features like zoom-on-scroll and better use of all mouse buttons could reduce reliance on the icons in the lower-right region, streamlining the user experience.

Generally, I think this is a great utility, but it still needs a few tweaks for professional daily use.

[loicknuchel]

I believe this utility (and similar ones) can be very useful when starting out.

Hi @nenad-vujicic thanks for your feedback :+1: Glad you found Azimutt very useful.

About your two issues:

1. I'm using Cytoscape.js for the auto-layouts, I don't know why some nodes still overlap sometimes, still investigating. But I'm not sure I can do much better and used algorithm are way more advanced than anything I can do myself ^^ On the bottom right, you can choose several kind of arrangements (cube icon), have you tried the Breadth one? For great layouts, they should be done manually, Azimutt can do that if you create an account and store it, but it's a paid feature.

2. the scroll interaction (vertical/horizontal) is mapped to moving in the view, for zooming you can use the ctrl+scroll one. There is many different expectations between PC/Mac users and also given the app people are used to, it's hard to the the perfect intuitive experience for everyone. Any specific shortcut you would like to have?

[nenad-vujicic]

1. I'm using Cytoscape.js for the auto-layouts, I don't know why some nodes still overlap sometimes, still investigating. But I'm not sure I can do much better and used algorithm are way more advanced than anything I can do myself ^^ On the bottom right, you can choose several kind of arrangements (cube icon), have you tried the Breadth one?

Yes, "breadth-based approach" looks better but it seems it doesn't implement another pass for eliminating overlapping. Perhaps adding new pass which will do "something like sphere packaging" could eliminate overlapping? I used 15y ago variation of something similar (but much harder to understand / implement) which worked perfectly on much harder data. Perhaps this can give you some ideas.

2. the scroll interaction (vertical/horizontal) is mapped to moving in the view, for zooming you can use the ctrl+scroll one. There is many different expectations between PC/Mac users and also given the app people are used to, it's hard to the the perfect intuitive experience for everyone. Any specific shortcut you would like to have?

Yes, sorry, Ctrl+Scroll & pinching on touchscreen work great, but pinching on laptop's mousepad is very slow. Also, it would be great if you could merge Drag tool and Select tool by doing selecting when user clicks on one of tables and panning when user clicks on edge or background (multi-select can be done by e.g. Ctrl + select ROI).

[loicknuchel]

I tweaked a bit the parameters to improve some layouts but it still can have some overlapping sometimes. I don't really have the time to read a 167 page thesis and try to implement their findings, do you know if there is reference implementations I can have a look?

What is the Ctrl + select ROI you are suggesting for the multi-select? That's the thing preventing me to do panning on background drag.

[nenad-vujicic]

I tweaked a bit the parameters to improve some layouts but it still can have some overlapping sometimes. I don't really have the time to read a 167 page thesis and try to implement their findings, do you know if there is reference implementations I can have a look?

There is a C++ / OpenGL implementation, but it's a little bit complex. Have you tried cytoscape-cola or some other library like D3 (both should have non-overlapping modes, not sure about implementations)?

What is the Ctrl + select ROI you are suggesting for the multi-select? That's the thing preventing me to do panning on background drag.

Sorry, I thought, what you do now (left-mouse-click + mouse move => draws rectangle and selects all tables from it), but just add additional constraint for that action like pressed Ctrl button.