search

webbukkit / DynmapBlockScan

Tool to scan post-1.8 MC blocks and JSON model data to automatically generate Dynmap models
61 stars 30 forks source link

Status of Block Scan #1

Open Lagosaurus opened 7 years ago

Lagosaurus commented 7 years ago

Hey Mike,

What's the current status of the new scraper? Been messing about with 1.10 packs and the missing Dynmap models are fairly extensive. I'd be happy to test things out if you need any help.

brad18 commented 7 years ago

I'd be happy to test things too. I can see it doing stuff on launch but can't seem find any info on what to do. Does it output a file?

mikeprimm commented 7 years ago

I'll be getting more active on this later this week - I've been in the middle of a 1.11.2 migration/upgrade for the server I admin (and do much of the custom development for), which has also served as 'research' to better understand how to do what I want to accomplish with the BlockScan code (which will ultimately be part of Dynmap - its just a research mod right now). Ultimately, I'm hopeful to be able to handle MOST of post 1.8 mods 'automatically', by parsing the block state and model data contained within the mods - but its a BIG piece of work, with a lot to happen before there is anything to test.

mikeprimm commented 7 years ago

OK - this now is DOING something :) It isn't even close to done, but many/most solid cube blocks provided by any mod should now render (including blocks like logs and such that have simple rotation patterns). To use try this out (it's only on Forge 1.11.2 right now):

I'm eventually planning on combining DynmapBlockScan into the base Dynmap v3.0, but for now, it's a whole lot more convenient to keep them separate (for dev/debug, and to avoid not breaking Dynmap too bad while its being developed).

At this moment, the code would probably work fine on Dynmap 2.5 or later, but I'll be starting to hack on the Dynmap 3.0 APIs to support the needs from the DynmapBlockScan publication of blocks (basically, that is what is happening here : DynmapBlockScan does analysis of blocks, their blockstate JSON data, their model JSON data, and the mapping of their BlockState definitions in order to 'derive' rendering models to publish to Dynmap through it's 'modsupport' API - which causes temporary texture and model definition files to be created (under 'renderdata/modsupport' directory). So, if any of you are familiar with those files, you can see what was mapped/published, and how, by looking there after the server has started.

NJDaeger commented 7 years ago

Will this be available for Bukkit as well?

mikeprimm commented 7 years ago

Eventually, sure - the 'handling custom block automatically' thing is a lot less interesting there, but the 'final' version will also include handling of custom block models in RPs (including vanilla), and should handle new vanilla blocks without explicit changes MOST of the time.

The latest version of DynmapBlockScan is, as of this morning, a LOT more functional: it should be doing a pretty fair job of handling both solid blocks, cuboid-style block, and patch-based blocks, which should be most blocks. Blocks that are not trivial metadata mapped (that is, one model per metadata value) will be funky at times (stairs, doors, fences, etc), but they should still render. Next big step will be tying in the custom blockstate mappers, which are needed for those kinds of blocks to work (and will ultimately be the limit on handling custom blocks with very odd state).

mikeprimm commented 7 years ago

I've added a 1.12 build - http://www.dynmap.us/builds/DynmapBlockScan/DynmapBlockScan-3.0-alpha-1-forge-1.12.jar for DynmapBlockScan, http://www.dynmap.us/builds/DynmapForge/Dynmap-HEAD-3.x-forge-1.12.jar for DynmapForge v3.x. These are even more lightly tested than the 1.11.2 - I'll still be doing most dev and testing on 1.11.2, since there are more mods there, but for anyone brave enough to want to give 1.12 a try, there you go!

mikeprimm commented 7 years ago

We now have builds for 1.10.2, 1.11.2, and 1.12 - all the latest can be found under http://www.dynmap.us/builds/DynmapBlockScan . All depend on the latest DynmapForge 3.x builds from under http://www.dynmap.us/builds/DynmapForge for the corresponding version.

durand1w commented 7 years ago

I deployed the 1.10.2 versions on our server. It seems like the fullrender completed far faster, but maybe that is just my perception. Most mod blocks look good, far better than the dead forest look we had.

The few issues I wanted to report are that it doesn't appear to apply the block scan to non-standard dimension....ex: MoCreatures Wyvern Lair DIM-17. The scan runs without issues, but only vanilla blocks show.

The other block type that 'might' be broadly applicable would be for plants using the crossed texture.

SalmonSays commented 7 years ago

I've got 3.0 alpha 1 of both BlockScan and Dynmap running on a 1.11.2 server; it's a modified version of ATM2 and I'm thrilled to see it's running as intended with working textures for important stuff like Biomes o' Plenty! It's really cathartic to see the proper terrain without large black spots that we'd gotten so used to on previous versions of Dynmap. The one important mod that didn't appear to stick during the scan is Chisel, any word on workarounds/support?

wunder-bread commented 7 years ago

Attempted the 1.10.2 versions of Dynmap & Block scan. Server loads about 75% and just shuts down. No crash log, and nothing in the logs. Just stops. Any guidance on next steps? Would be much appreciated. forge-1.10.2-12.18.3.2281-universal spongeforge-1.10.2-2281-5.2.0-BETA-2558

vScourge commented 7 years ago

I know it's getting older, but can we expect builds for MC 1.7.10? I could offer our server/players up for testing if that helps.

mikeprimm commented 7 years ago

It literally cannot work on pre-1.8 builds: the whole basis of BlockScan is to work off of the data-driven (JSON-encoded) rendering data for blocks that most blocks use, starting as of 1.8. Pre 1.8 block models are all 'in code'.

Zelow79 commented 6 years ago

Tried to run BlockScan on FTB Beyond 1.11 and 1.10 (both are MC version 1.10.2) each with previous installations and fresh installations, all resulted in the server crashing. I made an issue report here, https://github.com/webbukkit/DynmapBlockScan/issues/11.

Sixdd6 commented 6 years ago

Trying to use the alpha1 for 1.12.2 and getting a crash.

crash report server latest log

VLange80 commented 6 years ago

@mikeprimm Great work so far on block scan. A note, it's not rendering chisel blocks in 1.12.2. Modpack i am using (the one i am working on) is here: https://www.curseforge.com/minecraft/modpacks/chaos-portal-modpack edit: also my dynmap can be seen here: http://minecraft.chaosportalshow.com:8123

BillyGalbreath commented 6 years ago

Looks good so far. Only thing I've noticed not working (showing as black) are forge liquids.

mikeprimm commented 6 years ago

Chisel support is unlikely to come from DynmapBlockScan - anything that does custom renderers vs using the standard JSON-based model scheme is pretty much hand-coded support (and I really cannot find the time to reverse engineer Chisel and recreate their rendering logic in Dynmap...). There is still a load of work needed to finish the block state support for the stock blocks (stairs, fences, walls, doors, etc) that involve non-trivial logic (for things like figuring out which block state to use depending upon neighboring blocks) - that'll get priority until they are all done, since 90%+ of custom blocks use the JSON-based models and are either trivial or based on stock block behaviors.

ThaDaVos commented 6 years ago

@mikeprimm maybe contact the mod-authors, you'll never know if they are willing to help if you don't ask :D

ThaDaVos commented 6 years ago

Getting an exception when trying a fullrender: https://gist.github.com/dvdbot/fb05d6a01b0fabe56b372c95c2267046 Had to copy the console because the error isn't in the log

also tried one of the nether and the error may be a bit better: https://gist.github.com/dvdbot/c7d6fd2ddc2ec051d7a677fc8f7107f2 Also copy of console

EDIT I'm playing Forever Stranded Lost Souls - May be handy to know if you want to test yourself

ThaDaVos commented 6 years ago

I looked into your code :D and I think the following is null on line 1219 in DynmapCore/TexturePack.java : tp.tile_argb === null

ThaDaVos commented 6 years ago

Made my own issue: https://github.com/webbukkit/DynmapBlockScan/issues/19

ThaDaVos commented 6 years ago

Good news - Chisel will eventually be using full vanila json models :D https://github.com/Chisel-Team/Chisel/issues/656

Ashakiwi commented 5 years ago

Am I not looking in the right spot? I didn't see any 3.x versions of dynmap on the link provided. >.<

Routhinator commented 5 years ago

I keep getting this with Beta-1 + DynmapForge 3.0 beta 3

[05:12:54] [Server Watchdog/FATAL] [minecraft/ServerHangWatchdog]: A single server tick took 60.00 seconds (should be max 0.05)
[05:12:54] [Server Watchdog/FATAL] [minecraft/ServerHangWatchdog]: Considering it to be crashed, server will forcibly shutdown.

It happens as the blocks are initially scanned, producing many lines of this:

[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/endstone_2.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/endstone_3.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/endstone_4.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/netherbrick_0.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/netherbrick_1.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/netherrack_0.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/netherrack_1.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/marble_0.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/marble_1.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/marble_2.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/marble_3.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/marble_4.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/limestone_0.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/limestone_1.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/limestone_2.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/limestone_3.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/limestone_4.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/basalt_0.json : Failed to open blockstate
[05:12:53] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/basalt_1.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/basalt_2.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/basalt_3.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/basalt_4.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/plate_iron_0.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/plate_iron_1.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/bronze_0.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/tin_0.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_chisel:assets/corail_pillar_extension_chisel/blockstates/copper_0.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_quark:assets/corail_pillar_extension_quark/blockstates/quark_block_0.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_quark:assets/corail_pillar_extension_quark/blockstates/quark_block_1.json : Failed to open blockstate
[05:12:54] [Server thread/INFO] [DynmapBlockScan]: [DynmapBlockScan] corail_pillar_extension_quark:assets/corail_pillar_extension_quark/blockstates/quark_stained_planks_0.json : Failed to open blockstate

This is in 1.12.2

Routhinator commented 5 years ago

Was able to work around this by disabling max-tick-time in server.properties, but seems odd it should lock the thread that long. For those that need the setting: max-tick-time=-1

Fallouthero commented 5 years ago

Question, I ran this and it fixed almost everything. One major issue I'm running into is on Project Red Basalt blocks. They still won't render. Any ideas?

Routhinator commented 5 years ago

@Ashakiwi http://www.dynmap.us/releases/

Routhinator commented 5 years ago

@Fallouthero if they are showing the Failed to open blockstate message for the blocks on startup its because they are not using the JSON-based model scheme as Mike mentioned. That's a won't fix. Open a ticket with Project Red and ask them to use the standard model system to improve their compatibility with Dynmap

Ashakiwi commented 5 years ago

@routhinator I cannot give you enough <3s. Haha thanks

ghost commented 5 years ago

What about chisel ? I have the same issue with latest build, and they said me to come here :/

ThaDaVos commented 4 years ago

What's the progress on this? Currently download the latest DynMap Forge build from the builds repo version beta-6 for 1.12.2

Exymat commented 4 years ago

I'm interested too!

CryptoSiD commented 3 years ago

Hi!

I've been able to "compile" DynmapBlockScan for Forge 1.16.5 from this repo: https://git.kalka.io/kalka/dynmapblockscan I had to adjust the versions in some files but it compiled properly. However, it's not working properly with "All The Mods 6".

Looks like the mods rats and ratlantis are causing issues so I cloned the minecraft server, deleted the map, removed rats and ratlantis mods, and tried to starts the server with dynmapblockscan but it's still scanning after 8 hours! (the server have 64GB of memory and 24 core)

I also have this server that only has 48 mods and I've been able to do a full scan with dynmapblockscan: https://minecraft.donsid.net/ Some blocks are still not getting rendered properly though, there's a lot of errors during the scan.

Just wondering if anyone got this working properly with 1.16.5 and "All The Mods 6"?

And second, is there any plan to make this mod works properly with 1.16.5?

SalmonSays commented 3 years ago

Not to disparage the creator of Dynmap because I loved and used the mod for a long time, but after all the complexity of mod compat I gave up and moved on. Bluemap handles this so much better. Bluemap supports traditional 2d mapping, 3d mapping, and free look with or without forge on 1.16, and getting forge mods to work is as easy as putting ones you want mapped into a folder. If it's an option for you I'd seriously recommend checking it out.

On Sun, May 30, 2021, 4:31 AM CryptoSiD @.***> wrote:

Hi!

I've been able to "compile" DynmapBlockScan for Forge 1.16.5 from this repo: https://git.kalka.io/kalka/dynmapblockscan However, it's not working properly with "All The Mods 6".

Looks like the mods rats and ratlantis are causing issues so I cloned the minecraft server, deleted the map, and tried to starts the server with dynmapblockscan but it's still scanning after 8 hours!

I also have this server that only has 48 mods and I've been able to do a full scan with dynmapblockscan: https://minecraft.donsid.net/ Some blocks are still not getting rendered properly though, there's a lot of errors during the scan.

Just wondering if anyone got this working properly with 1.16.5 and "All The Mods 6"?

And second, is there any plan to make this mod works properly with 1.16.5?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webbukkit/DynmapBlockScan/issues/1#issuecomment-850962816, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZHZ3KFF3CRINH4QLUNIRDTQHZUJANCNFSM4DKMDQYA .