BurntSushi / erd

Translates a plain text description of a relational database schema to a graphical entity-relationship diagram.
The Unlicense
1.79k stars 154 forks source link

Any desire to make this a service? #1

Open howardroark opened 10 years ago

howardroark commented 10 years ago

I proposed the same idea in project today which takes a similar text based approach to rendering UML sequence diagrams as SVG. I personally feel that the ability to write fluidly about technical ideas in the form of markdown is a big help. Having a service that would let you render any type of visualization during your stream of conscious seems like it could be quite helpful.

The idea being something like this:

![INITIAL IDEA](http://service/
[Person]//
*name//
height//
weight//
+birth_location_id////
[Location]//
*id//
city//
state//
country//)]

This would output a PNG rendering of the image in the markdown.

This is the other project which sort of rides a similar idea this one... https://github.com/bramp/js-sequence-diagrams

BurntSushi commented 10 years ago

I think this is a great idea. I even have a web server lying around doing little more than serving my blog and homepage. So it'd be nice to put it to good use. :-)

However, the example you gave has some regrettable syntax. I love the idea of embedding it straight into a Markdown document, but is there a nicer way to write the schema description? (i.e., get rid of the / characters.)

BurntSushi commented 10 years ago

Note: I'd be willing to make small adjustments to the er syntax to make it more forgiving.

howardroark commented 10 years ago

JSON?

![SCHEMA](http://bit.ly/1h0dkcL?json={
  "Person": [
    "*name",
    "height",
    "weight",
    "+birth_location_id"
  ],
  "Location":  [
    "*id",
    "city",
    "state",
    "country"
  ]
})

![SCHEMA](http://bit.ly/1h0dkcL?json={ "Person": [ "name", "height", "weight", "+birth_location_id" ], "Location": [ "id", "city", "state", "country" ] })

I must confess that I am more of a big picture guy than a programmer, haha.

howardroark commented 10 years ago

Also, this is great service if you like the convenience of just having servers laying around...

http://www.cloudatcost.com/pricing.php

One time fee for VMs.

BurntSushi commented 10 years ago

Yeah, I'm on DigitalOcean right now.

JSON could definitely work.

akalra commented 10 years ago

Hey -- any updates on this?

Would love this too, for a lot of the same reasons. Prefer to keep ER and Sequence diagrams in text because they're easier to write/update (I don't want to figure out how to lay things out, etc), and can also be versioned more effectively.

Happy to help if there's an easy way for me to be useful. I'm a developer, but have little Haskell experience at this point.

Ankur

BurntSushi commented 10 years ago

@akalra I'm a little confused. erd already operates on a text format that can be checked into revision control. This issue is about adding a web service so that you can give it a URL (with the ER description in the URL) and get back an image.

It shouldn't be hard to do on your own if you really need it---there's no need to know Haskell. Just invoke the command line program.

One problem with this approach is that there is a limit on the size of the query string for a URL, so it would only be able to support shortish descriptions.

howardroark commented 10 years ago

Hey @BurntSushi ... If you are curious, I did some experimenting with this idea.

I used the tool js-sequence-diagrams, which is an awesome JS tool inspired by websequencediagrams.com. When it comes to making diagrams it just makes sense to use the DOM and all of the projects that are available. As a start it saves you dollars in server costs.

In Markdown it is a problem though because your only real option is an image tag. I decided to make a service which takes GET parameters as an array of strings and then takes a snapshot of a web document with PhantomJS. It was a pretty feeble attempt in terms of effort, but it does act as a proof of concept.

I put it up here: https://github.com/codingcoop/get-diagrams

I have also been thinking a lot about diagrams in Markdown. I have put forward an idea for a Macro spec under the "CommonMark" project @ jgm/stmd#123

akalra commented 9 years ago

Hey @BurntSushi -- sorry, was tired when I wrote that so it might not have come off super clearly.

Yes, I already use it as a text-to-image tool and love it. The reasons I listed are all part of why I love it, and part of why I think as many technical diagrams as possible should be generated from version-controlled textual descriptions.

In the interest of promoting that viewpoint, I'd be open to helping turn it into a service so that more people can use it without having to go through setting it up and installing it. Setting it up locally is useful for statically generating diagrams, but makes it hard to integrate it with a project wiki or as part of the build-process of a CI server or something.

Are you envisioning more of a websequencediagrams.com style usage (POST with body semantics), or more of a Google Charts type usage (GET with query-string semantics)? They both make sense for different things.

Ankur

BurntSushi commented 9 years ago

@akalra Sorry for the late reply! I think we got our wires crossed. I definitely wasn't questioning the motivation---a web service would be very cool. I was chewing on the implementation strategy. Putting the ER description inside the URL seems like a bad idea because of URL length limits. Which leaves putting it into the request body... but that kind of defeats the purpose of being able to conveniently write the ER description inside a Markdown description or something. See the dilemma?

akalra commented 9 years ago

@BurntSushi -- yeah, I think we're in violent agreement here. :)

Four half-thought-through implementation options, though none appear stellar:

Option 1 -- GET with query string semantics Not sure you can get around the body content problem with GET and query-string semantics. I did find this (somewhat dated) research: http://www.boutell.com/newfaq/misc/urllength.html

..that suggests that in practice, URL lengths that don't need to be rendered in the browser address bar can be quite long (100k+ chars), assuming you can find an HTTP server that will respect it (Apache circa 2006 capped at 4k chars, IIS at 16k).

In short, there's probably a workable-but-not-recommended solution there somewhere, though it sets up the wrong incentives.

Option 2 -- plain POST semantics As you point out, solves the body problem but rules out some interesting use-cases unless you do something out of band (include a tiny JS library of some sort or something).

Option 3 -- POST + GET POST the body, return a token, GET the token. Crappy because the server now needs to keep state, and it's not clear what lifecycle semantics to advertise (how/when do tokens expire, etc).

If the token is some encoded and highly compressed form of the document, you can get around the state problem a little bit, but it's still an awkward experience -- write your Markdown doc, transform into a token, inline an image with the token somewhere.

Option 4 -- GET + resource GET something where the only arg you pass in is a pointer to a publicly accessible document describing the structure. Probably some significant performance/security concerns on the server end, unless you tightly constrain the kinds of pointers you accept (eg: has to be a gist); and likely security concerns on the client end too -- requiring the doc to be publicly accessible might be a non-starter for some people.


If this were up to me, I'm not entirely sure what I'd do. Probably have Option 1 be the common path, with a fallback of Option 2 for longer documents. The rest seem considerably more complicated for not considerably more value.

ftomassetti commented 9 years ago

Any progress on that? Any thinking about creating a webserver ?

BurntSushi commented 9 years ago

No progress. It's something I'd like to do, but it requires a time budget that I just don't have. Certainly others should press on with it if they like!

ftomassetti commented 9 years ago

Ok, I will look into that and let you know id there is any progress. Ideally I would like something like http://www.planttext.com/planttext

I could write the syntax highlighting module for CodeMirror (and maybe autocompletion also).

howardroark commented 9 years ago

Hey @ftomassetti have a look at... https://bramp.github.io/js-sequence-diagrams/

It is very similar to planttext! I was playing using phantomjs to make it a web service here... https://github.com/codingcoop/get-diagrams

ftomassetti commented 9 years ago

I wrapped this in a web server (https://github.com/ftomassetti/erd-web-server), just in case someone is interested

@howardroark your stuff seems cool, but I was looking for ER diagrams

0xcaff commented 6 years ago

This could be implemented client side with wasm!

windbender commented 6 years ago

Cool Idea. checkout. https://www.websequencediagrams.com. for inspiration.

It would be double awesome if it was JS and therefore embeddable in web pages. But making Graphviz work in webasm, would be exciting to say the least.

whatacold commented 5 years ago

Hi,

Recently I take some time to implement a web server, https://texttoolkit.com/erd-diagram ( Yeah, the UI is ugly, I know :( ), I hope it will help someone.

There are times I think a web application will be more convenient:

Currently, it's pretty simple, here are some caveats:

mmzx commented 5 years ago

Thanks for letting us know! The resulted image can be downloaded when one views it in the browser.

gonzaloserrano commented 5 years ago

Hey @whatacold that's awesome, any plan to make it open source? Thx!

whatacold commented 5 years ago

@gonzaloserrano I would open source it if you're insterested, though it's quite simple :)

UPDATE: Here it is, https://github.com/whatacold/erd-repl .

gonzaloserrano commented 5 years ago

Thank you!

ggrossetie commented 1 year ago

https://kroki.io/ supports ERD:

[Person]
*name
height
weight
+birth_location_id

[Location]
*id
city
state
country

Person *--1 Location

https://kroki.io/erd/svg/eNqLDkgtKs7Pi-XSykvMTeXKSM1MzyjhKodQ2kmZRSUZ8Tn5yYklmfl58ZkpXFzRPlAeUAuQn5xZUslVXJJYksqVnF-aV1JUycUFMVJBS1fXUAGmGgCFAiQX