How to use this?

[Fabian42]

There are no instructions on how to install this library. I found this when running yay nbt on my Manjaro Linux system:

4 aur/python2-nbt 1.5.0-1 (+3 0.00%) 
    Named Binary Tag Reader/Writer
3 aur/python-nbt 1.5.0-1 (+7 0.00%) 
    Named Binary Tag Reader/Writer

But I have no idea if either of those two is this project here. And I couldn't make any changes to the code if I wanted to. It's also probably not in the default repositories on other distributions and especially not on Windows.

Simply running import nbt in Python without installing anything of course does not work.

[Fabian42]

Here are my experiences of trying to use the python-nbt package from that list:

[fabian@laptop Downloads]$ python
Python 3.8.1 (default, Jan 22 2020, 06:38:00) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import nbt
>>> nbtfile = nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/region/r.0.0.mca",'rb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'nbt' has no attribute 'NBTFile'
>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/region/r.0.0.mca",'rb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/nbt/nbt.py", line 613, in __init__
    self.file = GzipFile(filename, 'rb')
  File "/usr/lib/python3.8/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/home/fabian/hdd/drive/minecraft/saves/New\\ World\\ \\(5\\)/region/r.0.0.mca'
>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/region/r.0.0.mca",'rb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/nbt/nbt.py", line 628, in __init__
    self.parse_file()
  File "/usr/lib/python3.8/site-packages/nbt/nbt.py", line 652, in parse_file
    type = TAG_Byte(buffer=self.file)
  File "/usr/lib/python3.8/site-packages/nbt/nbt.py", line 99, in __init__
    self._parse_buffer(buffer)
  File "/usr/lib/python3.8/site-packages/nbt/nbt.py", line 105, in _parse_buffer
    self.value = self.fmt.unpack(buffer.read(self.fmt.size))[0]
  File "/usr/lib/python3.8/gzip.py", line 286, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.8/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.8/gzip.py", line 473, in read
    if not self._read_gzip_header():
  File "/usr/lib/python3.8/gzip.py", line 421, in _read_gzip_header
    raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b'\x00\x02')
>>> exit()

TLDR: Nothing works.

[macfreek]

Hi @Fabian42 , it's a library, so indeed you need to write your own program to use it. There is documentation at the wiki.

For example, you will read that this library has 4 modules:

• nbt.nbt module for parsing and generating the NBT (named binary tag) message format.
• nbt.chunk module for interpreting the resulting data structure into high-level function specific for Minecraft (note that this module is outdated, as Minecraft has been updated more often than this library)
• nbt.region module for reading a region file (.mca or .mcr files), which contain 32x32 = 1024 NBT-formatted "chunks".
• nbt.world module for reading a folder of multiple region files.

To get started, check out the example scripts in the examples directory.

Regarding the specific exception in your second comment, you are on the right track. Just need one more attempt :)

>>> import nbt
>>> nbtfile = nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/region/r.0.0.mca",'rb')

with import nbt indeed imports the whole library. In that case, you should access nbt.nbt.NBTFile instead of nbt.NBTFile, as you found. from nbt import nbt would also work.

>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/region/r.0.0.mca",'rb')

You are quoting the spaces in the filename. There is no need to do that. In fact, it will fail, as it will look for a file called "New\ World\ (5)/region/r.0.0.mca" instead of the file "New World (5)/region/r.0.0.mca".

>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/region/r.0.0.mca",'rb')

You are opening a .mca file, which is a region file (of type "Minecraft Anvil"). Such a region file contains multiple NBT message structures. This NBT library was written when Minecraft did not have such region files, but separate NBT files.

Also note that you don't need a 'rb' parameter, this is not a raw file open() command.

Minecraft has a few NBT files. For example:

>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/playerdata/45e9e86a-e825-4044-a28c-699e8d4bc0d8.dat")
>>> print(nbtfile.pretty_print())

(Replace the UUID with an existing player) This will print the details of a player file, which is a file with gzipped NBT data.

If you want to open a region file, use:

>>> r = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/region/r.0.0.mca")
>>> for chunk in r.iter_chunks():
>>>     print(chunk)

Or simply run one of the example scripts. For a good understanding of the region files, you can run ./examples/regionfile_analysis.py -v /home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/region/r.0.0.mca. To get started ./examples/biome_analysis.py /home/fabian/hdd/drive/minecraft/saves/New\ World\ \(5\)/ or one of the other example scripts.

[Fabian42]

Thanks, that's a lot of useful information! Some of it should probably go into Readme. But it doesn't answer the initial question: How would someone get the library from GitHub and use it? I was lucky to be running a system with pretty often occurring repository updates, but what if you update this library tomorrow and a Debian user wants to use it immediately? Or a Windows user? Or someone who is paranoid and wants to make sure that the code they see on GitHub actually matches the one they execute?

[Fabian42]

If I create the file test.py and write into it:

import nbt
nbtfile = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/test/region/r.0.0.mca")
for nbt in r.iter_chunks():
    print(chunk)

And then execute it from a console (just ./test.py), then my cursor changes into a plus and when I click anywhere, I get the following output:

/home/fabian/Downloads/test.py: line 2: syntax error near unexpected token `('
/home/fabian/Downloads/test.py: line 2: `nbtfile = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/test/region/r.0.0.mca")'

What the heck? Why did the cursor change and wait for me to click?

Also, when I execute the individual lines directly in a console, this happens (with some guesses for what you actually mean done by me):

[fabian@laptop Downloads]$ python
Python 3.8.1 (default, Jan 22 2020, 06:38:00) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import nbt
>>> nbtfile = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/test/region/r.0.0.mca")
>>> for nbt in r.iter_chunks():
...     print(chunk)
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'r' is not defined
>>> for nbt in nbtfile.iter_chunks():
...     print(chunk)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
NameError: name 'chunk' is not defined
>>> for nbt in nbtfile.iter_chunks():
...     print(nbt)
... 
{TAG_Compound('Level'): {15 Entries}, TAG_Int('DataVersion'): 2510}
{TAG_Compound('Level'): {15 Entries}, TAG_Int('DataVersion'): 2510}
{TAG_Compound('Level'): {15 Entries}, TAG_Int('DataVersion'): 2510}
[…maaany more lines of this…]

So I guess this goes into the right direction at least. But I couldn't find any further documentation for this library, that would allow me to proceed.

[Fabian42]

Here is something that looks promising, but apparently it just tells me how to install the wiki and dependencies, not the actual library itself: https://github.com/twoolie/NBT/blob/master/doc/documentation.rst

[Fabian42]

I found some documentation: https://github.com/twoolie/NBT/wiki/Chunk But I also cannot get that to work:

>>> for nbt in nbtfile.iter_chunks():
...     print(nbt.get_all_blocks_and_data())
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
AttributeError: 'NBTFile' object has no attribute 'get_all_blocks_and_data'

At this point, this might just be my lacking knowledge of Python, but I would have assumed nbt to be a chunk, not a file.

[Fabian42]

Apparently the plus shaped cursor creates a screenshot of the clicked-on window. It was definitely triggered by the script, because the screenshot ended up in the ~/Downloads folder.

[macfreek]

How would someone get the library from GitHub and use it?

To download: You can download a released version of the code. Since it is a git repository, you can also clone it. (E.g. git clone git@github.com:twoolie/NBT.git. Alternatively, install from one of the distributions. Since this library is also on PyPI, you can use pip: pip install NBT. To use it, well, as you did. You write a script or program that uses it. For example the Minecraft Region Fixer uses this library.

As for your question how to ensure you have the latest version: well, that's up to a package manager (in case of Debian, I don't think there is an apt package for this library, so you can use pip as the package manager). If you want to use the very last version (which may contains bugs), you can use git pull. This is not different from other software that you may install on your computer.

As for your question how to ensure validity of the software. To check if it is the same, you can check the SHA checksum of the distribution. PyPI publishes these (click the "view" button of the downloads page). Pip and git check these before installing. But the more fundamental question is: what makes you trust any software to run on your computer? That's a difficult question which I can't answer for you. Perhaps you trust github to delete this repository if it would contain malicious code.

> nbtfile = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/test/region/r.0.0.mca")
> for nbt in r.iter_chunks():
>     print(chunk)

As you noted, there are two minor errors in here: you assign the RegionFile object to the variable nbtfile, and then use the variable r, which is not defined. Similarly you loop over a variable called nbt, but print chunk. (That last one was an error in my code above, I fixed it, thanks!)

As for your other comments on this library, beware that this library has been written almost 10 years ago. There is some support, but not very active development. It could be improved, but so far no-one stepped forward to do that (it's a volunteer effort). In particular, I find two things a bit annoying:

• How to open and close a file is not always clear. E.g. the NBTFile.__init__() function behaves differently if you pass a stream, buffer or filename. A better design would be to make explicit functions like NBTFile.from_file(), NBTFile.from_stream(), etc. I haven't made these changes because it broke a few things and didn't find the effort worthwhile.
• The code sometimes mixes "NBT" (the generic data structure) and "Chunk" (the Minecraft-specific helper functions around a NBT structure for a piece of the Minecraft world called a "chunk"). For example, in some case, RegionFile.get_chunk() returns a NBTFile object, not a Chunk object. That's confusing. I once wrote some improvements, but never finished those, as I don't have an immediate need for this code at the moment.

That said, with all it's quirks, this is still my go-to library if I want to examine or edit a Minecraft world.

One thing I often do is to get to understand a library is by:

• try out in the console, and use tab-completion a lot. For example once you assigned a NBTFile object to a variable mynbt, type mynbt. (with the dot) and hit tab. You should see a list of methods associated with that object.
• Read the documentation - in this library, that's mostly a list of the comments in the code, condensed on a HTML page at the wiki
• Read the example scripts, and try to modify those to suit your need.

Enjoy, keep trying, and if you feel some things can be improved, discuss here if it's useful, and if so, make the change yourself. I'm happy to explain how to make a pull request at that time.

[Fabian42]

To download: You can download a released version of the code. Since it is a git repository, you can also clone it.

Yeah, sure, I know how to download files. But how would I install it then? Or otherwise access it from a script?

what makes you trust any software to run on your computer? […] Perhaps you trust github to delete this repository if it would contain malicious code.

I did not ask that. I asked how I would know whether the one in the repository is the same as here on GitHub or a completely unrelated program/library/package that happens to have the same name.

you assign the RegionFile object to the variable nbtfile, and then use the variable r, which is not defined

Yes, that was in your example as well. I fixed it myself right below there.

I think in general we're both not understanding each other properly, so I'll just say this: My end goal (at least for now) is just to get the contents of an NBT file (player.dat, region file, …) printed as readable text. I'll probably do actual programming with NBT later, but for now I would be happy to just be able to properly read an NBT file (and search in it with Notepad++). I was unable to find any tool that does this conversion/printing. I tried all the ones listed on the Minecraft wiki: https://minecraft.gamepedia.com/index.php?title=NBT_format&oldid=1481941#Third-Party After I crossed off all tools that simply did not work at all, only NBTExplorer was left, which does not export to text. I also tried to do it with the data generator integrated into Minecraft servers, this went similarly well: https://bugs.mojang.com/browse/MC-173446 This library is currently my only hope left for doing this, apart from setting up MCP in an IDE and analysing and changing Minecraft code for weeks.

[macfreek]

Hi @Fabian42, we indeed seems not to understand each other properly.

I guess I'm a bit confused that I interpret your question as "how would I install the library, or otherwise access it from a script?". But to me, it seems that you already did it from the start: you did write code that uses this library. So that's how.

About how you know this code here on github is the same library as the code in some package found elsewhere on the internet: I don't think there is a definitive answer to that, but don't regard that as an issue myself. When I check for a package in my favourite package manager, I manually search for the package, and check if that's the right one by reading the package description. If so, I install it. If not, I look further.

As for a way to print the NBT content as readable text, use the pretty_tree() method to get a rought understanding of the contents of the files. For example:

>>> nbtfile = nbt.nbt.NBTFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/playerdata/45e9e86a-e825-4044-a28c-699e8d4bc0d8.dat")
>>> print(nbtfile.pretty_tree())

(Replace the UUID with an existing player)

and

>>> r = nbt.region.RegionFile("/home/fabian/hdd/drive/minecraft/saves/New World (5)/region/r.0.0.mca")
>>> for nbtfile in r.iter_chunks():
>>>     print(nbtfile.pretty_tree())

However, be aware that this will print out a LOT of information. Easily many MByte, and that it will still not print all details (arrays of numbers are not printed in full). However, it does give a good starting point, allowing you to find what you are looking for. The scripts in the example directory give you an idea what to do when looking for a particular piece of information.

[Fabian42]

I guess I got too much general support mixed into this ticket. Please ignore everything I said so far that included any code. My initial question was that simple: How to take this thing from GitHub and use it?

Windows doesn't have a package manager, Debian's packages are often very outdated and anyone can publish a package and write anything into its description. For example the Telegram messenger is not officially in any repositories for Debian-based systems, yet you can find a package there that claims to be official.

And what if I didn't like how this library used the name "pretty_tree" for a method and instead wanted it to use the name "hello_lalala" locally on my own copy? I couldn't do that with something coming from a package manager, I would need to edit the code and then somehow install that modified library.

What I am expecting as an answer is a series of commands or steps that gets me from "downloaded zip file of this project" to "something usable in a Python program". For example for many programs it's "./configure && make && sudo make install" or "unzip and then run this .sh script" or whatever. But there are zero instructions here for what to actually do once I have downloaded the project.

[macfreek]

How to take this thing from GitHub and use it?

Just like every other of the thousands of Python libraries on Github. Honestly, this information is in most tutorials on Python or git, and is not specific for this particular library. Here is the summary.

Method A) Visit https://pypi.org/, search NBT, and you'll see that this library has a pip package. Install using pip install nbt.

Method B) Download from github: git clone git@github.com:twoolie/NBT.git.

Really, that's all there is to it. You already did that.

And what if I didn't like how this library used the name "pretty_tree" for a method and instead wanted it to use the name "hello_lalala" locally on my own copy? I couldn't do that with something coming from a package manager, I would need to edit the code and then somehow install that modified library.

Yes, your assertion is correct. If you want to modify the library, you would need to edit the code, and wouldn't be using a vanilla version coming from a package manager.

If you want to make modifications to the code, you can do that on your local computer. If you want to publish those modification, you can do so too. (The license allows you to republish modified works). Click the fork button on the top right of this page.

I don't mean to shoo you away, but this is basics of how git and github works. Please try, get started, and enjoy. I think you're already well underway.

[Fabian42]

Again, I know how to download (or use Git to download). I'm asking about the step after the download. I'm also not asking about package managers.

I found this: https://packaging.python.org/tutorials/installing-packages/#installing-from-local-archives

From that I would assume that the installation instruction are these:

1. Download+unzip or Git clone.
2. pip install --no-index ./whatever/path (pointing to top level folder of the project)

Is that correct?

[macfreek]

Never used that command. Just pip install nbt, or if there is no PyPI package, cp -r to the site-packages directory.

My apologies, Fabian, but I seem to fail to understand your goal or question. These are all basic things that are not NBT-library specific. I really tried to help, but I think I'm not able to help you anymore than what I did, so hopefully this helped. If not, I suggest asking someone who can better help you. My recommendation is elsewhere, as your questions do not seem specific to this library. I hope you nevertheless appreciated my attempt at answering you.

[Fabian42]

pip install nbt installs it from a repository, not a local folder. I want to actually use the code that I have on my computer!

Do you mean I should just copy the downloaded project into /usr/lib/python3.8/site-packages? Or maybe only the "nbt" folder inside it? But there also seems to be a .egg-info folder for every library listed in there, where would I get that from?

Giving installation instructions is a basic thing that should be in every single program's/library's readme file. Even if you have done this a million times, it's not at all guaranteed that everyone knows how to install your software.