Graphics.Draw renders Verticies wrong unless Material Editor or any Scene editor is open

Describe the bug

This is a weird one.

When using Graphics.Draw or Vertexbuffer.Draw methods after selecting a Material in the AssetBrowser the Locations of the Verts is wrong unless opening a Editor with a Scene in it so: Hammer with Map, ModelDoc, Material Editor with Compiled Material.

This persists until closing the game even between restarts. Not selecting anything in the Assetbrowser will display it normally. then breaks when selecting a Material.

Also Drawing without ever having a Scene editor open also doesn't work, might be related to my game not having any SceneObjects in it so it never initialized the Attributes correctly?

To Reproduce

Download and Launch example-test.zip
if everything appears normal, restart the game. (normal means a Grid on the ground)
if just a whitescreen appear this is the Initial bug of not drawing.
Click on worldmat.vmat or any compiled material
see the Grid being angled
Open the vmat or ModelDoc
see it being normal again
closing it will make it break again

Example Project: example-test.zip

(drawing is done in the MainGameHook.cs file at the very bottom)

Expected behavior

obviously it should draw normally like it does when opening ModelDoc.

Media/Files

Example of it being broken:

https://user-images.githubusercontent.com/37816311/206924346-fa51e2a5-b4de-453a-b63f-2112e8a37912.mp4

Additional context

No response

Facepunch / sbox-issues