File names encoding issues (virtualenv issues)

[Brottweiler]

Trying to run from source, but virtualenv ENV gives me trouble:

Using base prefix '/usr'
New python executable in ENV/bin/python3
Also creating executable in ENV/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /home/brott/Download...fied/ENV/bin/python3 -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
  File "/usr/lib/python3.5/site-packages/virtualenv_support/pip-7.1.2-py2.py3-none-any.whl/pip/__init__.py", line 13, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
  File "/usr/lib/python3.5/site-packages/virtualenv_support/pip-7.1.2-py2.py3-none-any.whl/pip/utils/__init__.py", line 23, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
  File "/usr/lib/python3.5/site-packages/virtualenv_support/pip-7.1.2-py2.py3-none-any.whl/pip/_vendor/pkg_resources/__init__.py", line 30, in <module>
  File "/home/brott/Downloads/MCEdit-Unified/pkgutil.py", line 545
    except IOError, msg:
                  ^
SyntaxError: invalid syntax
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 3, in <module>
    virtualenv.main()
  File "/usr/lib/python3.5/site-packages/virtualenv.py", line 832, in main
    symlink=options.symlink)
  File "/usr/lib/python3.5/site-packages/virtualenv.py", line 1004, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python3.5/site-packages/virtualenv.py", line 969, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python3.5/site-packages/virtualenv.py", line 910, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/brott/Download...fied/ENV/bin/python3 -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel failed with error code 1

What is up?

[LaChal]

No problem !

Please, can you post the log file? It contain information I need...

[Brottweiler]

Ah sorry, here: https://gist.github.com/4b9ec05880e1aca3aed5

[LaChal]

Thanks !

The log does not signal anything strange...

You told that ~/.mcedit/MCEdit/Schematics was empty. (It shall not crash MCEdit...) If still empty, can you populate it with the needed files, either by switching install mode or copying the content of stock-schematics into ~/.mcedit/MCEdit/Schemetics. If not empty, please mind to rename the files you have added in this directory if any. Especially if you downloaded some from the web. It may happen that the files dowloaded can contain "corrupted" characters. You can first keep thier original names, but if it does not work try with Latin base letters. If it still does not work and the directory is not empty, please provide the list of the files which ar in there. (And the links to the dowloaded files, if any.)

---

I also run on Linux. Looking to the tracebacks an logs you sent, there's not so much differences between your's and mine. (Same system encoding (UTF-8), same file system encoding (UTF-8 too), and a lot of other common points.) Except files names in the scematics directory, I can't see what's wrong.

---

Another thing.

Can you tell me what graphical client do you use (Gtk, KDE,...) ? It may help adding support ;)

[codewarrior0]

@LaChal Do you remember when I asked you to create a file whose name is just the byte 0xA0? If you do that you should be able to reproduce this issue.

The problem is that os.listdir will return a mix of unicode and str if one of the files has a filename that cannot be decoded. This causes os.walk to attempt to use os.path.join on the unicode dirname and the str basename of the filename that cannot be decoded.

[Brottweiler]

Yep, it was empty. I just copy pasted the stock schematics, and it still crashes. My "proof":

$ ls -al ~/.mcedit/MCEdit/Schematics
total 16
drwxr-xr-x 4 brott brott 4096 Oct 20 23:06 .
drwxr-xr-x 6 brott brott 4096 Oct 16 08:12 ..
drwxr-xr-x 2 brott brott 4096 Oct 20 23:06 Mobs in Jars
drwxr-xr-x 2 brott brott 4096 Oct 20 23:06 Spawners

This might be pointless for me to put here considering what codewarror wrote but doing it anyway.

[codewarrior0]

Would you mind doing a find .mcedit/MCEdit/Schematics too?

[Brottweiler]

$ find .mcedit/MCEdit/Schematics | gist --private --anonymous

https://gist.github.com/9dd2b739195b75281c8e

[codewarrior0]

Oh, I see. It isn't even looking in .mcedit/MCEdit/Schematics. When it opens the Choose File dialog, it first opens it to <cwd>, and then afterwards changes its directory to .mcedit/MCEdit/Schematics, so it crashes when it lists the files in <cwd>.

Check the working directory where you run mcedit for files with unusual names.

[Brottweiler]

This is just getting more and more confusing.

So in that folder I had some weird files with question marks in them. My file manager said they had the wrong encoding or something, whatever, I deleted them. MCEdit would now crash on launch, so I thought fuck it I just redo it.

This is my progress:

1. git clone --recursive git@github.com:Khroki/MCEdit-Unified.git
2. sudo pacman -S sdl_ttf (This package seems required since it would crash with and error otherwise)
3. cd MCEdit-Unified
4. virtualenv2 ENV
5. . ENV/bin/activate
   1. pip install PyOpenGL
   2. pip install numpy==1.9
   3. pip install pygame==1.9.1release --allow-external pygame --allow-unverified pygame
   4. pip install pyyaml
   5. pip install Pillow
   6. pip install ftputil
   7. python2 mcedit.py

Now, I see the splash screen, but I get a new crash error:

Traceback (most recent call last):
  File "mcedit.py", line 73, in <module>
    import keys
  File "/home/brott/Downloads/MCEdit-Unified/keys.py", line 4, in <module>
    import albow
  File "/home/brott/Downloads/MCEdit-Unified/albow/__init__.py", line 22, in <module>
    from albow.file_dialogs import \
  File "/home/brott/Downloads/MCEdit-Unified/albow/file_dialogs.py", line 26, in <module>
    from tree import Tree
  File "/home/brott/Downloads/MCEdit-Unified/albow/tree.py", line 15, in <module>
    from extended_widgets import ChoiceButton
  File "/home/brott/Downloads/MCEdit-Unified/albow/extended_widgets.py", line 4, in <module>
    import resource_packs
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 914, in <module>
    packs = ResourcePackHandler()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 869, in __init__
    self._resource_packs = setup_resource_packs()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 843, in setup_resource_packs
    frp = FolderResourcePack(folder_tex_pack)
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 744, in __init__
    self.add_textures()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 796, in add_textures
    self.parse_terrain_png()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 622, in parse_terrain_png
    print "An Exception occurred while trying to parse textures for {}".format(self._pack_name)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa7' in position 0: ordinal not in range(128)

[LaChal]

Thank you for your effort @Brottweiler. Sorry for the inconvenience.

Your installation process is not the source of the UnicodeError crashes you're experimenting.

The 'ascii' codec can't decode byte 0xff in position 1 is related to the question-marked files you mentionned. The new one is related to the names of textures of one of your Minecraft resourcepack.

AFAIK, resources packs can be folders or .zip files. You may have the same "question mark" issue in one of them. To solve this error, move your texture packs one after another. So you'll can find whic one(s) cause the new issue.

Sorry to ask you to test every resource pack, but I can see anoter bug in the traceback...

---

@codewarrior0: the mixed character sets was one of the cause I thought about, but without any clue, it was difficult to diagnose precisely. Thank you for your help about this :)

I thought that <cwd> was a safe place to initialize the component, but it is clear it is not. I'll find something else safer.

[Brottweiler]

Alright, everything seems to work now. You were right;

In my resourepack folder I had one folder (I have all my packs in folders, not .zips) called §6§lSummerFields§r_McP§r_1.8.X. Notice the § signs, which makes color codes. This is just to make the name fancy in the RP selection menu, which you probably know. I renamed the folder to remove the § symbols, and now mcedit starts!

I also pressed "Import", and it opened up a integrated file browser! Thanks for your help. :stuck_out_tongue:

[LaChal]

All right ! Nice to read you :)

But the last crash you had can be fixed if you mind to test again with the guilty resource pack. (In fact the code shall skip the "unreadable packs" , but it obviously don't...)

If you want to test just pull from GitHub and restore the rsource pack. You shall have 'errors' in the console.

[Brottweiler]

I just git pull, and then I renamed the RP again to include the § symbol, and when I launch mcedit I get this:

Running in fixed mode. Support files are in your Documents folder.
Splash load...
Traceback (most recent call last):
  File "mcedit.py", line 13, in <module>
    import splash
  File "/home/brott/MCEdit-Unified/splash.py", line 22, in <module>
    splash = pygame.image.load(open(splash_name).read().strip())
pygame.error: Couldn't open /home/brott/Downloads/MCEdit-Unified/splashes/splash0.png

I then renamed the folder again to remove the § symbol, and it still crashes with the same error.

[Brottweiler]

Oh damn... it's because I moved the directory... It doesn't like that...

[Brottweiler]

Ok, trying again MCEdit launches but still throws an error due to the resourepack name;

An Exception occurred while trying to parse textures for u'\xa71SummerFields_McP_1.8.X'
  File "mcedit.py", line 73, in <module>
    import keys
  File "/home/brott/Downloads/MCEdit-Unified/keys.py", line 4, in <module>
    import albow
  File "/home/brott/Downloads/MCEdit-Unified/albow/__init__.py", line 22, in <module>
    from albow.file_dialogs import \
  File "/home/brott/Downloads/MCEdit-Unified/albow/file_dialogs.py", line 26, in <module>
    from tree import Tree
  File "/home/brott/Downloads/MCEdit-Unified/albow/tree.py", line 15, in <module>
    from extended_widgets import ChoiceButton
  File "/home/brott/Downloads/MCEdit-Unified/albow/extended_widgets.py", line 4, in <module>
    import resource_packs
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 919, in <module>
    packs = ResourcePackHandler()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 874, in __init__
    self._resource_packs = setup_resource_packs()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 848, in setup_resource_packs
    frp = FolderResourcePack(folder_tex_pack)
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 749, in __init__
    self.add_textures()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 801, in add_textures
    self.parse_terrain_png()
  File "/home/brott/Downloads/MCEdit-Unified/resource_packs.py", line 628, in parse_terrain_png
    traceback.print_stack()
Exception Message: bad transparency mask
Exception type: <type 'exceptions.ValueError'>

[LaChal]

Hmmm...

This time, the name of the pack is not involved.

The texture format in causing this. Texures are images and they are sort of 'encoded' to manage color sets and transparency.

I guess if you move this pack (and any other casting the same error), all will be fine. But you will not be able to use them in MCEdit...

If you mind to post the link(s) to the file(s) you donwloaded, we'll have a look. (You're not the first one who have this type of issue.)

[Brottweiler]

It's not really an issue anymore, just rename it and it'll be fine. In the first line it clearly sais u'\xa71SummerFields_McP_1.8.X'. It's this pack but you could just rename any RP you got and just add § or something.

[LaChal]

The first line is fine now :) The error is casted by another part of MCEdit.

Do you experience a crash? (It shall not...)

[Brottweiler]

No, as I said MCEdit started and I could load a world and everything, but the console still threw that error above. MCEdit did not crash.

[Podshot]

Did you move the pack (zipped or folder) out of your resourcepacks folder?

[Brottweiler]

I just renamed it to remove the § symbol, which works.

[Brottweiler]

Actually this is weird. MCEdit works and all, but as I said above, I renamed the folder to remove the § symbols, but MCEdit still throws an error even when there's no special symbols;

An Exception occurred while trying to parse textures for SummerFields_McP_1.8.X
  File "mcedit.py", line 73, in <module>
    import keys
  File "/home/brott/minecraft/MCEdit-Unified/keys.py", line 4, in <module>
    import albow
  File "/home/brott/minecraft/MCEdit-Unified/albow/__init__.py", line 22, in <module>
    from albow.file_dialogs import \
  File "/home/brott/minecraft/MCEdit-Unified/albow/file_dialogs.py", line 26, in <module>
    from tree import Tree
  File "/home/brott/minecraft/MCEdit-Unified/albow/tree.py", line 15, in <module>
    from extended_widgets import ChoiceButton
  File "/home/brott/minecraft/MCEdit-Unified/albow/extended_widgets.py", line 4, in <module>
    import resource_packs
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 919, in <module>
    packs = ResourcePackHandler()
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 874, in __init__
    self._resource_packs = setup_resource_packs()
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 848, in setup_resource_packs
    frp = FolderResourcePack(folder_tex_pack)
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 749, in __init__
    self.add_textures()
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 801, in add_textures
    self.parse_terrain_png()
  File "/home/brott/minecraft/MCEdit-Unified/resource_packs.py", line 628, in parse_terrain_png
    traceback.print_stack()
Exception Message: bad transparency mask
Exception type: <type 'exceptions.ValueError'>

I just re-setup the installation cause I wanted to move folders. Just FYI if you notice the path's are different from before.

Edit: The second time I launched MCEdit I didn't get the error above, lol.

[LaChal]

This message is given when MCEdit can't load one of the textures of a resource pack. It is no more displayed on next MCEdit startups because the textures of resource packs are cached in files in a directory in your MCEdit install.

[Brottweiler]

Ah. I'm not using anything else other than default, so. :stuck_out_tongue:

[LaChal]

@Brottweiler: The issue concerning the texture loading shall no more happen now. If you want to test, just pull this repo, delete the "SummerFields" PNG file you can find in MCEdit-Unified/terrain-textures and run MCEdit. You shall be able to select this resource pack from the Graphics menu.