Can't create connection to container or objects with 'near' placement control set, results in "err: failed to compile: 153:3: cannot connect objects from within a container, that has near constant set, to objects outside that container"

[HariSekhon]

I've got a diagram of OpenTSDB on HBase and I've set near: bottom-center on the Hadoop/HBase cluster because it must be underneath the OpenTSDB cluster like so:

But I want to connect the OpenTSDB nodes to the HBase nodes, but any connection to the cluster or cluster nodes such as:

tsdb1 -> hadoop

results in this error:

err: failed to compile: 153:3: cannot connect objects from within a container, that has near constant set, to objects outside that container

If I comment out the near: bottom-center the the diagram comes out messed up with the Hadoop/HBase cluster being put out to the side (the very reason I came to D2 running away from Python diagrams module in the first place)

The full code is here in my Diagrams-as-Code github repo:

https://github.com/HariSekhon/Diagrams-as-Code/blob/master/opentsdb_kubernetes_hbase.d2

[alixander]

you can try the following together

1. direction: down
2. invisible connection between the two
3. use ELK

(3) only because dagre's container-container connection isn't great and looks a little weird.

https://play.d2lang.com/?script=tJnfjpw2FMbveYojrXKXBcx_uIjUNEq3qtRVlfR6xILD0p0Bansmiap598oGsxg7hu2qV9mxv-_453PsMwxhLTviAv5xAESuCrgfcPf504f30Hdw976k2AHgOSqA9cNthTuGiQMg8lEAX5p_4tDuXLcCIn8enRcvgJEzdq6OM520cd0zxWT6E4AXsIBHxgZaeB7_RF2GCSkpI-eKuVV_8jCluGNteaRvgo9hnN-KCC69NA7AleM2bfftJRFrfHkTfBS2RZinjB7opTIHqtuyIeWJuqe243-cMBGh2lPjkemgUO8po16H2deePHn0UrlDJ4P3A-4YrR_U6K7XnsoGU09OLxyPDyXFUn6zwhGTbjmU1SN2e9LIOGL8cOyb_vC1ZY-HnlTlFPN1QQ5HfiRfHOqv82lou-ZWhCA94-1pDvJfMtx3A8Enry5ZyZecFl5kraz7fnhNDacVyq48fmdtRb0x5LzG1XFu4I9z2THqioNdjEfauYEP-GIaux-00Xv2iMn4SZ2aY8PtO34i3d_5KYVfu4ZgSuEXUn4pu3Jaa4fmfthULVjsUme6uiPgW4HwdlrkLfw5bW3clNZCnKeMFvDb-QGTDjNMoe9ueaJFpQ7uvLxx6QJCOLXHY9t38PcZkxZTGDCBuvzugNki0ypuuSaSXW-tmsxAMbm0FZbTU2tw4Lld_lBhpOE7W8XWV1uKhr4G5P4kLhTcMTbUgLYdgeoIHAfg88_98Ygr1hM4YUbaikLZ4MXplb3HptUrMyfi9p2WlAJyeD-VS8ZpuwZThmtZt-mGjt8Z8sll-srg12yxWZELeaXVlLivuOGyUU9dS3YlueqrYp_4c0lP2q7xmuk8yvjqBnj6GrliAU-YPGDSUxhI_-371NPUMzHzmTK_TltQqB9NSQz-xyS-KvbeJAbLJAY7khjMSQx-lMT1GNfxS4Isc4FlLrTMRZa52DKXWOZSy1xmmctte_dtk7bMIFtqkC03yJYcFPNSib8WD7JDX6PpnOsd7joZAtUQbBpC1RBuGiLVEG0aYtUQbxoS1ZBsGlLVkG4aMtWQbRpy1ZBvGpC_qpy_bVkXe7vaaFVutF1vtCo42q44WpUcbdccrYqObFV3-HO3cPELcHDHB9N5NDCOhsbRyDgaG0cT42hqHM2Mo7lxFPnmYfPukHl7yLw_ZN4gUndoTqjb9TUG5IqfFsimCUZNYNOEoya0aaJRE9k08aiJbZpk1CQ2TTpqUpsmGzWZTZOPmtyaQ39Kom9VyVRbc42mZIuvkpsXVS7YUTmDRqucQaNVzqDRKmfQaJUzaLTKGTRa5QwarXIGjaFyJpVeOZPquXJXx5E_0O_Ev1Adz5RhovwaHF3ygX_0FuPLKZC9fu6QYloMXUd5_YWiOfzdh4-fDB7ZFWaTWIOjC7t8ryROisIRSI5gD0egcugeI0cgOYIlR6hwhJIj3MMRqhy6x8gRSo5wyREpHJHkiPZwRCqH7jFyRJIjWnLECkcsOeI9HLHKoXuMHLHkiJccicKRSI5kD0eicugeI0ciOZIlR6pwpJIj3cORqhy6x8iRSo50yZEpHJnkyPZwZCqH7jFyZJIjW3LkCkcuOfI9HLnKoXuMHLnkyJcc_AF32cj8uZP5u1qZv-plusvczPy5m_kKzqqvPjfWfZ113Vr39tbn5qp0V6S2VzT3V7SrwaJVhzW4zDhzj0WBfKn7lIm3XMsXyOPLqem_iQrwnavzbwAAAP__&layout=elk&

[HariSekhon]

I had tried direction: down initially as one of the first things, but it didn't work, and nor does the invisible connection between the two, it's only ELK that makes it work when combined with both of the other two options.

I expect it's difficult to fix near and connections to work together?

That placement control given by near is such a key feature... as are connections... having to choose between the two or switch engines entirely seems like a pretty big limitation?

Thank you for the workaround though, I'll try that.

[HariSekhon]

So since raising this ticket, I switched all the icons to shape: image... which looked better, but when comined with --layout elk just now, results in the arrows not reaching the icons any more, look at the middle of this diagram, the two arrows around the OpenTSDB service go into thin air (probably because there would have been a big horizontal rectangle with a little icon in the middle without shape: image:

[HariSekhon]

This is the best I've been able to come up with by going back to the default dagre layout engine, using near placement control of the hadoop cluster and not using any connections between the tsdb objects or k8s container and the hadoop container, due to the original error message from this issue:

[switchupcb]

This issue could be fixed if the use of grid on the blank canvas is implemented (https://github.com/terrastruct/d2/issues/1145).

The user could specify:

# title, first node, kubernetes on prem, hadoop cluster on prem
grid-columns: 4

# prevent nodes from appearing on right or left side
grid-rows: 1

# Influence the diagram to declare nodes in a specific direction (from declaration order?)
direction: down

Documentation https://d2lang.com/tour/grid-diagrams/ https://d2lang.com/tour/layouts/#direction

[HariSekhon]

Hitting this issue with another diagram too:

https://github.com/terrastruct/d2/discussions/1416

[HariSekhon]

I think this placement control is a really big issue, it affects this one too:

https://github.com/terrastruct/d2/discussions/1300

[AndreiPashkin]

Very strange - this code doesn't work with v0.6.1 CLI, but does work in the Playground.

A: {
  near: top-center
  a
  b
}
B: {
  x
  y
}
A.a -> B.x

https://play.d2lang.com/?script=crRSqOZSUAA50UqhJL9ANzk1ryS1iEtBIZFLQSGJq5bLCaKigktBoZKrlstRL1FB107BSa-CCxAAAP__&

[alixander]

@AndreiPashkin oh Playground runs on latest master branch. I made an issue to note that in Playground so it's clear: https://github.com/terrastruct/d2-playground/issues/42

[AndreiPashkin]

@AndreiPashkin oh Playground runs on latest master branch. I made an issue to note that in Playground so it's clear: terrastruct/d2-playground#42

Okay, my sanity is restored.