palewire / django-bakery

A set of helpers for baking your Django site out as flat files
https://palewi.re/docs/django-bakery/
MIT License
405 stars 61 forks source link

InvalidCharsInPath error for BUILD_DIR while attempting to build project #134

Open Krammy opened 5 years ago

Krammy commented 5 years ago

I'm getting an error with InvalidCharsInPath while trying to retrieve the BUILD_DIR.

In settings.py: BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) BUILD_DIR = os.path.join(BASE_DIR, 'builds')

I can't think of why it would be failing to recognise characters. The characters in the path specified by the error message don't seem to be any issue. This bug remains with or without the forward slash after builds.

Every time I use the manage.py build command, this error shows up:

Traceback (most recent call last): File "F:\Users\Mark\Documents\GitHub\Portfolio\portfolio\manage.py", line 15, in <module> execute_from_command_line(sys.argv) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line utility.execute() File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\management\__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\management\base.py", line 316, in run_from_argv self.execute(*args, **cmd_options) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\management\base.py", line 353, in execute output = self.handle(*args, **options) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\bakery\management\commands\build.py", line 97, in handle self.set_options(*args, **options) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\bakery\management\commands\build.py", line 143, in set_options if not self.fs.exists(self.build_dir): File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\fs\base.py", line 489, in exists self.getinfo(path) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\fs\osfs.py", line 264, in getinfo _path = self.validatepath(path) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\fs\osfs.py", line 643, in validatepath return super(OSFS, self).validatepath(path) File "F:\Users\Mark\AppData\Local\Programs\Python\Python36-32\lib\site-packages\fs\base.py", line 1475, in validatepath raise errors.InvalidCharsInPath(path) fs.errors.InvalidCharsInPath: path 'F:\Users\Mark\Documents\GitHub\Portfolio\portfolio\builds' contains invalid characters

I'd appreciate any help!

Krammy commented 5 years ago

The same issue arises while trying to build the example project.

fsecada01 commented 5 years ago

Can confirm that I am also running into this.

python manage.py build
Traceback (most recent call last):
  File "manage.py", line 24, in <module>
    main()
  File "manage.py", line 20, in main
    execute_from_command_line(sys.argv)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\django\core\management\__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\django\core\management\base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\django\core\management\base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\bakery\management\commands\build.py", line 97, in handle
    self.set_options(*args, **options)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\bakery\management\commands\build.py", line 143, in set_options
    if not self.fs.exists(self.build_dir):
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\fs\base.py", line 491, in exists
    self.getinfo(path)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\fs\osfs.py", line 273, in getinfo
    _path = self.validatepath(path)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\fs\osfs.py", line 661, in validatepath
    return super(OSFS, self).validatepath(path)
  File "C:\dev\Django Projects\FJS_Services_Site\lib\site-packages\fs\base.py", line 1477, in validatepath
    raise errors.InvalidCharsInPath(path)
fs.errors.InvalidCharsInPath: path 'C:\dev\Django Projects\FJS_Services_Site\src\storage/private/static_build/site/build/' contains invalid characters

settings file (inherits from base settings file)

from .base import *
import os

INSTALLED_APPS += [
    'bakery',
]

BUILD_DIR = os.path.join(BASE_DIR, 'storage/private/static_build/site/build/')

BAKERY_VIEWS = [
    'app_views.views.HomeView',
    'app_views.views.HomeToBeginView',
    'app_views.views.ServicesView',
    'app_views.views.UseCasesView',
    'app_views.views.AboutMeView',
    'app_views.views.ContactView',
    'app_views.views.SuccessView',
]
palewire commented 5 years ago

Is there a unicode or special character in your file name?

fsecada01 commented 5 years ago

There is not.

fsecada01 commented 5 years ago

Even if there was, there should be some code to filter that out. Something like:

file_path_str = [insert string code here]
file_path = file_path_str.encode('ascii', 'ignore').decode('unicode_escape')
palewire commented 5 years ago

Hmm. Any idea what's causing it? Can you give me some more information about your config or data inputs that might help us sleuth this out?

fsecada01 commented 5 years ago

I believe this is a coding error on part of the module. My configurations are fairly standard for a Django project. In the base setting file, I define the base directory as follows:

BASE_DIR = os.path.dirname(os.path.dirname(os.path.dirname(
    os.path.abspath(__file__))))

This allows me to refer to file and folder locations using relative paths, freeing me to develop within Windows and Linux environments.

palewire commented 5 years ago

I don't doubt there's a bug, I'm just struggling to find it so we can craft a patch. All of my tests are currently passing. So I suspect there is something I'm missing.

What is the path that is failing?

palewire commented 5 years ago

Are all of you guys on Windows?

fsecada01 commented 5 years ago

I am on Windows 10. But my system should be OS-agnostic.

Krammy commented 5 years ago

I'm on Windows 10. Maybe it's got something to do with my storage drive being F instead of C? That's the only reason I can think of why it's complaining of invalid characters in the path. I have two drives, one for my Windows installation which is an SSD, and one for general storage which is an HDD.

jaycle commented 5 years ago

Just to throw my voice into the mix. I'm experiencing this issue, too. Python 3.7 in virtual environment on, windows 10.

I also can't seem to get a relative path off the project to write where I want. Specifying ./build not only tries to write to C:\build it also tries to find the static files at /static when they're in <PROJECTDIR>/static.

EDIT: I don't know enough about osfs but just playing around a bit I noticed that changing the string osfs:/// (triple slash) to osfs:// got it at least using relative instead of off the root (C:\ in my case). I'll dig a bit more.

EDIT 2: Okay so it looks like PyFilesystem doesn't accept any non-forward slash delimited paths, by design. Since django-bakery wants to support multiple filesystems (through BAKERY_FILESYSTEM) we would need to map all Windows paths into the unix styled ones before use. I could take a stab at a PR if this is expected to affect many people and would be useful. If not, I'll probably just run things from within a Docker container.

danizen commented 4 years ago

@jaycle, in my government workplace, most developers are expected to work on Windows, and Docker is not permitted nor is WSL (Windows sub-system for Linux). A pull request would enable me to use django-bakery for a number of projects rather than django-distill. I prefer to user interface of django-bakery because a lot less boilerplate will be needed.

danizen commented 4 years ago

This line, https://github.com/datadesk/django-bakery/blob/2c9c495e4e8faca7b81fa57635d1631933f14171/bakery/views/base.py#L150, is problematic. If PyFilesystem will not allow backslash, then paths must be concatenated without using os.path.separator.

weilingwei commented 3 years ago

fs.errors.InvalidCharsInPath: path 'F:\workplace\scrapysite\wlwcms\builds' contains invalid characters good product~