Failed building for the second try

[amber-moe]

I'm following your guide in #21. When I run mkdocs build, I got the site directory as follows:

404.html       assets         search         sitemap.xml.gz
RC3 1.0.0      index.html     sitemap.xml

I then run mkdocs gh-deploy, but got a different site from expectation. There is no version selector and the version in the nav is 404.

Then I tried to build for the second time. The returned messages are:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/bin/mkdocs", line 10, in <module>
    sys.exit(cli())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/mkdocs/__main__.py", line 163, in build_command
    ), dirty=not clean)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/mkdocs/commands/build.py", line 298, in build
    config['plugins'].run_event('post_build', config)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/mkdocs/plugins.py", line 94, in run_event
    result = method(item, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/mkversion/entry.py", line 45, in on_post_build
    version(config, self.config)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/mkversion/version.py", line 47, in version
    os.remove(f.path)
PermissionError: [Errno 1] Operation not permitted: '/Users/zhangying/Desktop/doc-115/site/search'

I tried with both python3 and the virtual environment but the same error happens. You can find my project here.

[zayd62]

if you take a look at my test repo https://github.com/zayd62/mkdocs-versioning-test and the built docs https://zayd62.github.io/mkdocs-versioning-test/. lets compare my urls with your urls

1. https://zayd62.github.io/mkdocs-versioning-test/
2. https://amber1990zhang.github.io/doc-115/

opening up 1 will take you to the version selection page where as your one (2) will take you to the actual docs

next, lets take a look at the built docs deployed in github pages

1. https://github.com/zayd62/mkdocs-versioning-test/tree/gh-pages
2. https://github.com/Amber1990Zhang/doc-115/tree/gh-pages

taking a look at your mkdocs.yml

  - mkdocs-versioning:
      version: RC3 1.0.0

there is no folder in 2. called RC3 1.0.0

So it seems to be that the plugin is not making the version selection page.

There is also the same permission error which you are getting.

Hopefully, over the weekend I'll take a deep dive into your issue now that I have access to your markdown files

[zayd62]

Hi @Amber1990Zhang, see #40

[zayd62]

Hi @Amber1990Zhang, I released a new version of the plugin 0.2.0 on pypi https://pypi.org/project/mkdocs-versioning/. install it in a virtual environment and hopefully, it should fix your problem

[ppanero]

Hello,

With 0.2.0 I hitted this problem too. I am willing to contribute, if required. What is the status of the project (I saw #40, will check there for updates)?

[zayd62]

Couple of questions

1. Are you on Mac os
2. Can you provide the traceback here
3. Can you give me your directory structure from the root of your project
4. Can you provide your mkdocs.yml here Me and @timvink both had working

[ppanero]

@zayd62 Thanks for the fast response:

1. Yes OS X (Catalina 10.15.3)

2. INFO    -  Cleaning site directory 
   INFO    -  Building documentation to directory: /Users/ppanero/Workspace/inveniosw/docs-invenio-rdm/site/0.7.1 
   WARNING -  A relative path to '../' is included in the 'nav' configuration, which is not found in the documentation files 
   Traceback (most recent call last):
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/bin/mkdocs", line 8, in <module>
   sys.exit(cli())
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/click/core.py", line 829, in __call__
   return self.main(*args, **kwargs)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/click/core.py", line 782, in main
   rv = self.invoke(ctx)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
   return _process_result(sub_ctx.command.invoke(sub_ctx))
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
   return ctx.invoke(self.callback, **ctx.params)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/click/core.py", line 610, in invoke
   return callback(*args, **kwargs)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/mkdocs/__main__.py", line 159, in build_command
   build.build(config.load_config(**kwargs), dirty=not clean)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/mkdocs/commands/build.py", line 298, in build
   config['plugins'].run_event('post_build', config=config)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/mkdocs/plugins.py", line 96, in run_event
   result = method(**kwargs)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/mkversion/entry.py", line 44, in on_post_build
   version(config, self.config)
   File "/Users/ppanero/.virtualenvs/docs-invenio-rdm/lib/python3.6/site-packages/mkversion/version.py", line 47, in version
   os.remove(f.path)
   PermissionError: [Errno 1] Operation not permitted: '/Users/ppanero/Workspace/inveniosw/docs-invenio-rdm/site/search'

I tried giving permissions, even unsecure 777 but didnt work.

3.

% ls -la
total 48
drwxr-xr-x  13 ppanero  staff   416 Mar 26 17:49 .
drwxr-xr-x  13 ppanero  staff   416 Mar 25 09:22 ..
drwxr-xr-x  15 ppanero  staff   480 Mar 26 17:35 .git
-rw-r--r--   1 ppanero  staff   421 Mar 16 18:00 .travis.yml
-rw-r--r--   1 ppanero  staff  1107 Mar 16 18:00 LICENSE
-rw-r--r--   1 ppanero  staff   566 Mar 16 18:01 README.md
drwxr-xr-x  10 ppanero  staff   320 Mar 24 17:59 docs
-rw-r--r--   1 ppanero  staff     0 Mar 26 17:50 mkdocs.test.yml
-rw-r--r--   1 ppanero  staff  2172 Mar 26 17:49 mkdocs.yml
-rw-r--r--   1 ppanero  staff   285 Mar 16 18:00 requirements.txt
-rw-r--r--   1 ppanero  staff   400 Mar 16 18:00 run-tests.sh
drwxrwxrwx   8 ppanero  staff   256 Mar 26 17:50 site
drwxr-xr-x   3 ppanero  staff    96 Mar 16 18:00 theme

# Project information
site_name: 'User Documentation'
site_description: 'InvenioRDM user documentation web site'
site_url: 'https://inveniordm.docs.cern.ch'

# Repository
repo_name: 'docs-invenio-rdm'
repo_url: 'https://github.com/inveniosoftware/docs-invenio-rdm'

# Copyright
copyright: 'Copyright © 2019-2020 CERN and Northwestern University.'

# Configuration
edit_uri: '/0.7.0'

theme:
  [...]

nav:
  - Home: 'index.md'
  - [...]
  - Version: '../'

# Customization
extra:
    [...]
extra_css:
  - stylesheets/extra.css

# Extensions
markdown_extensions:
  - admonition
  - codehilite:
      guess_lang: false
  - toc:
      permalink: true

plugins:
  - mkdocs-versioning:
      version: 0.7.1

Note that I added [...] to make parts shorter where it was just a matter of more items but the same level.

Thanks a lot!

[zayd62]

so the problem i have is that i dont have a mac. I on linux/windows so im going to need your help on this.

testing environment

1. brand new virtual environment,
2. clone the github repository and install the plugin from the cloned github repository using pip install -e /path/to/cloned/repo.
3. run pip llist to make sure that the plugin is installed. pip list should show a bunch of packages including mkdocs and the plugin should come up something like this.

 mkdocs-versioning  0.2.0               /media/zayd/Common/Projects/mkdocs-versioning

your path is going to be different

debugging

i use visual studio code for debugging so i can only give instructions for vs code.

1. open up vscode and bring up the command pallete https://code.visualstudio.com/shortcuts/keyboard-shortcuts-macos.pdf --> and type python select interpreter and select the testing environment which you made above
2. in vscode click on the run icon on the side (4th one on the left) and press create a launch.json file
3. select module and type mkdocs. the plugin works by hooking into the standard mkdocs build pipeline
4. in launch.json, add "args": ["build"]. file should something look like this

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Module",
            "type": "python",
            "request": "launch",
            "module": "mkdocs",
            "args": ["build"]
        }
    ]
}

5. now you can add breakpoints, run the vscode debugger and inspect the individual variables and their values throughout the programs.

permissions

on Linux, I am able to build perfectly my docs perfectly fine. able to produce several different versions as well. as far as i know, mac os and linux permissions work the same way.

1. is your site folder empty. if yes then that means you are building for the first time. if no, then you have built before. you must then copy everything in the site directory into a folder eg: 0.7.0 and try and rebuild

start fresh

now that we have a clean environment and are able to make changes and debug the plugin as we go along, run mkdocs new . which should generatemkdocs.yml and /docs/ in the same directory. add your previous .md files to the docs folder and configure the plugin and the nav in mkdocs.yml

run mkdocs build and you should have your first versioned docs built alongside a version selection page in /site/. you can run python -m http.server to serve the built docs in the /site/ directory. the site directory should look something like this

.
├── 0.1.0
│   ├── 404.html
│   ├── assets
│   ├── CLI commands
│   ├── Getting Started
│   ├── index.html
│   ├── search
│   ├── sitemap.xml
│   └── sitemap.xml.gz
├── 404.html
├── assets
│   ├── fonts
│   ├── images
│   ├── javascripts
│   └── stylesheets
├── index.html
├── search
│   └── search_index.json
├── sitemap.xml
└── sitemap.xml.gz

[ppanero]

Hello @zayd62 thanks a lot for the detailed explanations. I will also try on linux. Just one thing:

is your site folder empty. if yes then that means you are building for the first time. if no, then you have built before. you must then copy everything in the site directory into a folder eg: 0.7.0 and try and rebuild

So currently my /site folder looks like:

├── 0.7.0
│   ├── 404.html
│   ├── assets
│   ├── deployment
│   ├── develop
│   ├── extensions
│   ├── images
│   ├── index.html
│   ├── install
│   ├── preview
│   ├── sitemap.xml
│   ├── sitemap.xml.gz
│   └── stylesheets
├── 404.html
├── assets
│   ├── fonts
│   ├── images
│   ├── javascripts
│   └── stylesheets
├── index.html
├── search
│   └── search_index.json
├── sitemap.xml
└── sitemap.xml.gz

So as you can see there is a 404, index, assets etc outside my 0.7.0 version folder. When you say take everything and move it into a folder. Does it mean that I should grab all that and move it to 0.7.0? So that my directory looks like:

└── 0.7.0
    ├── 0.7.0
    │   ├── 404.html
    │   ├── assets
    │   ├── deployment
    │   ├── develop
    │   ├── extensions
    │   ├── images
    │   ├── index.html
    │   ├── install
    │   ├── preview
    │   ├── sitemap.xml
    │   ├── sitemap.xml.gz
    │   └── stylesheets
    ├── 404.html
    ├── assets
    │   ├── fonts
    │   ├── images
    │   ├── javascripts
    │   └── stylesheets
    ├── index.html
    ├── search
    │   └── search_index.json
    ├── sitemap.xml
    └── sitemap.xml.gz

Doing so works, as in it can build. But then there is no versioning.

[zayd62]

Theres no need to movebl, looks good to me. Try serving the current directory with python -m http.server should bring up a version selection page

[ppanero]

Try serving the current directory with python -m http.server should bring up a version selection page

Yes, this worked. It worked also with mkdocs serve for me. But just with one version. Need to manage to build the second. Gonna try in linux and then debug OS X thanks!

[ppanero]

@zayd62 I tracked the issue and I know the error (I think). In order to provide a fix I want to make sure I understand the logic:

In version.py (line aprox 44):

        for f in files:
            if not f.is_dir() or (f.name in item_to_delete):
                try:
                    os.remove(f.path)
                except IsADirectoryError:
                    shutil.rmtree(f.path)

The error comes from trying to delete search which is a directory. Then the logic here says:

If it not a directory or I have to delete then delete

• Is it possible the case where there are file not in the item_to_delete and we want to delete? If not the if-then-else chain can be simplified. Otherwise, for OS X it works just by adding:

                except (IsADirectoryError, PermissionError):
                    shutil.rmtree(f.path)

The main issue is that os.remove raises a PermissionError in OS X instead of a IsADirectoryError. From what I see on the web... it is a common issue.

Now the issue is the "version dropdown is not populated" and the version page doesn't follow the styling of the others. But those are other issues. I will keep debugging! Thanks for the help!

[zayd62]

├── 0.7.0
│   ├── 404.html
│   ├── assets
│   ├── deployment
│   ├── develop
│   ├── extensions
│   ├── images
│   ├── index.html
│   ├── install
│   ├── preview
│   ├── sitemap.xml
│   ├── sitemap.xml.gz
│   └── stylesheets
├── 404.html
├── assets
│   ├── fonts
│   ├── images
│   ├── javascripts
│   └── stylesheets
├── index.html
├── search
│   └── search_index.json
├── sitemap.xml
└── sitemap.xml.gz

Taking a look at this, every thing apart from 0.7.0 needs to be deleted. That code deletes all the files and the folders assets and search. The reason for that is that is after the deletion, all that should be left is the previous built docs which is then used to build the version page

[zayd62]

Just saw your update on os X and permissions error. Try adding that in and see if it builds

[ppanero]

@zayd62 yes, it does work with:

  except (IsADirectoryError, PermissionError):
                    shutil.rmtree(f.path)

Then deploying on gh-pages too (https://ppanero.github.io/docs-invenio-rdm/)

I will test now if our docs work properly on it. Thanks a lot!

Thanks!

[zayd62]

deploy is done slightly differently https://zayd62.github.io/mkdocs-versioning/0.1.0/CLI%20commands/

[ppanero]

Yeah I saw. And it also works!

If you implement:

 except (IsADirectoryError, PermissionError):
                    shutil.rmtree(f.path)

I think you can close the issue. I can create a PR, just let me know.

[zayd62]

I made a branch permission-error so feel free to make a pull request.

The main issue is that os.remove raises a PermissionError in OS X instead of a IsADirectoryError. From what I see on the web... it is a common issue.

This is an interesting issue that I was not aware of. Being OS X specific explains why I could not reproduce it. perhaps in the code leave a little explanation about catching PermissionError and a link to this issue as a comment

Now the issue is the "version dropdown is not populated" and the version page doesn't follow the styling of the others. But those are other issues. I will keep debugging! Thanks for the help!

not sure about the dropdown issue nor the styling issue. This plugin is designed to be as "simple" as possible. the version page is actually built using the same code that mkdocs uses to build the actual site and the same mkdocs.yml file (if i remember correctly).

[timvink]

Good catch!

If it helps, github actions supports running unit tests on different OS versions (see an example here). I found some errors that way.

Shall I create a separate PR to setup github actions to run unit tests?

[zayd62]

Yeah. Was thinking about GitHub actions. I have some ideas about some more tests and some actions eg upload to pypi. I'll start another issue on that. Once @ppanero uploads his fix this issue can finally be closed

[timvink]

I added a simple start with pyflakes in https://github.com/zayd62/mkdocs-versioning/pull/44/files

[ppanero]

I cannot request reviews, but here you have the PR: https://github.com/zayd62/mkdocs-versioning/pull/45

cc @zayd62 @timvink

thanks!

[zayd62]

fixed in #45