search

ultrabug / mkdocs-static-i18n

MkDocs i18n plugin using static translation markdown files
https://ultrabug.github.io/mkdocs-static-i18n/
MIT License
219 stars 37 forks source link

Issue with incorrect language folder being copied #288

Open snowCho opened 5 months ago

snowCho commented 5 months ago

The contents of the mkdocs.yml file are as follows:

''' site_name: 'Guide' site_description: Documentation docs_dir: docs/IDE/

extra_css:

theme: name: material features:

extra: social:

markdown_extensions:

plugins:

The folder structure is as follows:

docs / ko / index.md ko_folder / ko_index.md en / index.md en_folder / en_index.md ja / index.md ja_folder / ja_index.md '''

When you do mkdocs build, the ko folder is copied to each of the en and ja folders in the site folder. ex) site / ko_index.md a folder / ko_index.md en / en_index.md en_folder / en_index.md ko_folder / ko_index.md ja / index.md ja_folder / ja_index.md ko_folder / ko_index.md

What's the problem?

kamilkrzyskow commented 5 months ago

Hello @snowCho, could you please check with plugin version 1.2.0? Perhaps my file lookup fix in 1.2.1 broke this. ๐Ÿค” Also as a side note Python 3.7 is quite old, consider updating to 3.12 this won't help with the bug, but should improve performance etc.

kamilkrzyskow commented 5 months ago

Yep, this was probably unnecessary in the fix commit: https://github.com/ultrabug/mkdocs-static-i18n/blob/d172407b6bd31b5363a9c7566a2724ad1164a5c8/mkdocs_static_i18n/folder.py#L146 The reversed loop probably causes the default to be fetched before the current language, and all languages end up as the default ๐Ÿ˜ฎโ€๐Ÿ’จ There has to be a reason why it didn't happen when I tested it previously ๐Ÿค”

EDIT: Investigating further, running the docs from https://github.com/ultrabug/mkdocs-static-i18n/issues/277#issuecomment-1791039178 together with plugin version 1.2.1 it turned out that there is mixed behaviour. A correctly resolved path which I focused on in https://github.com/ultrabug/mkdocs-static-i18n/pull/278

path='de_CH/legal/website.md'
current de_CH
default en
reversed(expected_src_uris):
0 de_CH\legal\website.md
1 en\de_CH\legal\website.md
2 en\legal\website.md

and a incorrectly resolved path:

path='bot/selfhosting.md'
current de_CH
default en
reversed(expected_src_uris):
0 bot\selfhosting.md
1 en\bot\selfhosting.md
2 de_CH\bot\selfhosting.md

it's as if the expected_src_uris are created differently, so the reversed also returns different results. I will have to think about it some more, how to best to resolve this, but probably paths with the current language prefix should be prioritized.

ultrabug commented 5 months ago

Damn indeed I thought my current tests would catch that but they didn't :( sorry.

@kamilkrzyskow should we revert and release 1.2.3 while a proper fix is found?

kamilkrzyskow commented 5 months ago

This is completely my fault, I've written in #278 that there is still some issue lingering, but I ignored the gut feeling and didn't properly squash the issue, and in turn prevent this current issue.

As for the release do as you like @ultrabug I won't be able to fix it today, should be possible tomorrow, I think that reverting releasing 1.2.2 and then releasing 1.2.3 with a fix is reasonable if deleting the pypi version isn't possible ๐Ÿค” As for the proper fix I thought about removing the relativity checks and just hardcoding this order always:

0 current-lang-prefix / path
1 path
2* default-lang-prefix / path

EDIT: Correction default-lang-prefix / path would only be added if the fallback_to_default is True

ultrabug commented 5 months ago

I dropped the 1.2.1 from PyPI, seemed the fastest.

We'll have to get correct tests up for the fix.

snowCho commented 5 months ago

Thank you for your quick reply. Let's guess the plugin version.

snowCho commented 5 months ago

-> Successfully installed mkdocs-static-i18n-1.2.0

I have the same problem with i18n 1.2.0 version. Just in case, I will install it in my local environment and test it.

แ„‰แ…ณแ„แ…ณแ„…แ…ตแ†ซแ„‰แ…ฃแ†บ 2024-01-31 แ„‹แ…ฉแ„Œแ…ฅแ†ซ 11 20 00

kamilkrzyskow commented 5 months ago

@snowCho Oh, I'm surprised that the issue is in 1.2.0 this could mean that the issue was there from 1.0.0 ๐Ÿค” Please check it with PR with pip install git+https://github.com/kamilkrzyskow/i18n.git@fix/folder-with-tests Also it would be great if you could show the debug information for a path that was incorrectly copied for the different languages:

path='de_CH/legal/website.md'
current de_CH
is_relative_to(de_CH\legal\website.md, de_CH) -> True
default en
is_relative_to(de_CH\legal\website.md, en) -> False
expected_src_uris:
0 de_CH\legal\website.md
1 en\legal\website.md
2 legal\website.md

this could perhaps explain some more about the issue ๐Ÿค”

snowCho commented 5 months ago

I'm sorry. I'm not a developer. First of all, I placed docs_folder_structure_two_languages โ€‹โ€‹of i18n GitHub in docs.

en_folder fr_folder

mkdocs.yml was modified as follows: `site_name: 'Guide of goormIDE' site_description: Documentation for goorm docs_dir: docs/

copyright: Copyright ยฉ 2024 goorm

theme: name: material

features:
  - content.tooltips
  - navigation.tracking
  - navigation.expand
  - search.highlight
  - search.suggest
  - search.share

markdown_extensions:

plugins:

build page

T_T)

snowCho commented 5 months ago

I ran the above command.

(venv) goorm@SnowCho mkdocs % pip install git+https://github.com/kamilkrzyskow/i18n.git@fix/folder-with-tests Collecting git+https://github.com/kamilkrzyskow/i18n.git@fix/folder-with-tests Cloning https://github.com/kamilkrzyskow/i18n.git (to revision fix/folder-with-tests) to /private/var/folders/_b/1yxyry390sqffzg4q_52r_hm0000gn/T/pip-req-build-di0d72pf Running command git clone --filter=blob:none --quiet https://github.com/kamilkrzyskow/i18n.git /private/var/folders/_b/1yxyry390sqffzg4q_52r_hm0000gn/T/pip-req-build-di0d72pf Running command git checkout -b fix/folder-with-tests --track origin/fix/folder-with-tests Switched to a new branch 'fix/folder-with-tests' branch 'fix/folder-with-tests' set up to track 'origin/fix/folder-with-tests'. Resolved https://github.com/kamilkrzyskow/i18n.git to commit d4e477cc8ff52182c557673531021ffd0c5333ae Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: mkdocs>=1.5.2 in ./venv/lib/python3.12/site-packages (from mkdocs-static-i18n==1.2.1) (1.5.3) Requirement already satisfied: click>=7.0 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (8.1.7) Requirement already satisfied: ghp-import>=1.0 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (2.1.0) Requirement already satisfied: jinja2>=2.11.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (3.1.3) Requirement already satisfied: markdown>=3.2.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (3.5.2) Requirement already satisfied: markupsafe>=2.0.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (2.1.4) Requirement already satisfied: mergedeep>=1.3.4 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (1.3.4) Requirement already satisfied: packaging>=20.5 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (23.2) Requirement already satisfied: pathspec>=0.11.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (0.12.1) Requirement already satisfied: platformdirs>=2.2.0 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (4.2.0) Requirement already satisfied: pyyaml-env-tag>=0.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (0.1) Requirement already satisfied: pyyaml>=5.1 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (6.0.1) Requirement already satisfied: watchdog>=2.0 in ./venv/lib/python3.12/site-packages (from mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (3.0.0) Requirement already satisfied: python-dateutil>=2.8.1 in ./venv/lib/python3.12/site-packages (from ghp-import>=1.0->mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (2.8.2) Requirement already satisfied: six>=1.5 in ./venv/lib/python3.12/site-packages (from python-dateutil>=2.8.1->ghp-import>=1.0->mkdocs>=1.5.2->mkdocs-static-i18n==1.2.1) (1.16.0) Building wheels for collected packages: mkdocs-static-i18n Building wheel for mkdocs-static-i18n (pyproject.toml) ... done Created wheel for mkdocs-static-i18n: filename=mkdocs_static_i18n-1.2.1-py3-none-any.whl size=21089 sha256=e550b6eced85445d6181939876922edc53c60e970a7440ed132a07b5ae21cfc4 Stored in directory: /private/var/folders/_b/1yxyry390sqffzg4q_52r_hm0000gn/T/pip-ephem-wheel-cache-ot10bm85/wheels/f2/af/08/bd8f42d79f74e31430fd89dcedbaf7e8558ac9faab813aca09 Successfully built mkdocs-static-i18n Installing collected packages: mkdocs-static-i18n Attempting uninstall: mkdocs-static-i18n Found existing installation: mkdocs-static-i18n 1.2.0 Uninstalling mkdocs-static-i18n-1.2.0: Successfully uninstalled mkdocs-static-i18n-1.2.0 Successfully installed mkdocs-static-i18n-1.2.1 (venv) goorm@SnowCho mkdocs % ls
docs mkdocs.yml site venv (venv) goorm@SnowCho mkdocs % mkdocs build INFO - mkdocs_static_i18n: Building 'en' documentation to directory: /Users/goorm/Desktop/mkdocs/site INFO - mkdocs_static_i18n: Adding 'fr' to the 'material/search' plugin 'lang' option INFO - Cleaning site directory INFO - Building documentation to directory: /Users/goorm/Desktop/mkdocs/site INFO - mkdocs_static_i18n: Selected en en File(src_uri='en/index.md', dest_uri='index.html', name='index', url='./') INFO - mkdocs_static_i18n: Selected en en File(src_uri='en/english_default/index.md', dest_uri='english_default/index.html', name='index', url='english_default/') INFO - mkdocs_static_i18n: Building 'fr' documentation to directory: /Users/goorm/Desktop/mkdocs/site/fr INFO - mkdocs_static_i18n: Selected fr fr File(src_uri='fr/index.md', dest_uri='fr/index.html', name='index', url='fr/') INFO - mkdocs_static_i18n: Selected en en File(src_uri='en/english_default/index.md', dest_uri='fr/english_default/index.html', name='index', url='fr/english_default/') INFO - mkdocs_static_i18n: Selected fr fr File(src_uri='fr/french_only/index.md', dest_uri='fr/french_only/index.html', name='index', url='fr/french_only/') INFO - Documentation built in 0.49 seconds

fr_folder_2 fr_page

After installation I built mkdocs and checked the fr folder. The en document is still in fr.

kamilkrzyskow commented 5 months ago

I'm sorry. I'm not a developer.

No worries, @snowCho you're doing great. I saw that after installing the PR version you don't have the log messages, this showed me an error on my part as I limited the print output to only 2 specific paths ๐Ÿ˜“ if path in ("de_CH/legal/website.md", "bot/selfhosting.md"):, so of course you don't have any additional debug information, as you don't have those paths. Sorry for the confusion. It would make it easier to read if you just attached the mkdocs.yml file directly instead of copying the text as the formatting breaks, or perhaps you're not using 3 ticks ``` to create code blocks, a single tick ` isn't enough for multiline code. Or zip the directory if possible and attach it here for investigation, but first with the new information provided I have a few comments:

Video of the built site https://github.com/ultrabug/mkdocs-static-i18n/assets/34622465/a646ec3f-e56d-49d5-a095-b2b0abbd41cf

At the end I switched from French to English and the path used was en/, because you configured it like that with the link /en/, but the plugin doesn't use the link to change the path of the files it uses it only for the alternate language switcher creation. Perhaps this could be seen as a bug, but I'm not so sure.

Please provide more information as after you accessed fr/ wanting it to be English I feel like there might be some misunderstanding ๐Ÿค” I've updated the PR branch, so you can reinstall it 

pip uninstall mkdocs-static-i18n
pip install git+https://github.com/kamilkrzyskow/i18n.git@fix/folder-with-tests

Then with the docs_folder_structure_two_languages please run:

mkdocs build > prints_i18n.log
mkdocs build > full_i18n.log 2>&1

Then with your own Korean Guide of goormIDE documentation please run:

mkdocs build > prints_goorm.log
mkdocs build > full_goorm.log 2>&1

Those last commands should create log files you can attach to the message / drag & drop will work as well. debug-files.zip

kamilkrzyskow commented 5 months ago

@snowCho any progress on those log files? I would also like to ask you to make a diff compare between the built files, because I can't reproduce the error of wrong files being served on the server as you said. The issue wasn't in https://github.com/ultrabug/mkdocs-static-i18n/issues/288#issuecomment-1915252000 because after fixing it, the result is the same on my end. I definitely need more information to find out where the issue is ๐Ÿ˜ž.

snowCho commented 5 months ago

@kamilkrzyskow I'm sorry. I wasn't able to try it yesterday because I was busy with work! I had no idea there were English documents in the French folder! I'll try it right now and get back to you!

snowCho commented 5 months ago

@kamilkrzyskow Here it is. Thank you! docs_folder_structure_two_languages log [full_i18n.log](https://github.com/ultrabug/mkdocs-static-i18n/files/14132773/full_i18n.log) [prints_i18n.log](https://github.com/ultrabug/mkdocs-static-i18n/files/14132777/prints_i18n.log)

goorm docs log full_goorm.log prints_goorm.log

[

แ„‰แ…ณแ„แ…ณแ„…แ…ตแ†ซแ„‰แ…ฃแ†บ 2024-02-02 แ„‹แ…ฉแ„Œแ…ฅแ†ซ 10 12 45

](url)

kamilkrzyskow commented 5 months ago

I forgot to mention that the PR got a little fix, after I asked you for the logs ๐Ÿ˜“, but it doesn't matter in this scenario. Comparing your prints_i18n.log with the one from my debug-files.zip I get an almost perfect match, minus my Windows paths and 2 more links I had in a file, however this in-part proves the behaviour is the same on my PC and yours. I didn't yet check the goorm.log too deeply yet, but at a first glance it does seem fine. I don't see a bug, but see room for improvement for the URL creation.

I think there is in-fact a misunderstanding here. I worked with the assumption that the wrong directory is being copied over into the site after build, but now I'm connecting the dots, and I understand your issue is with the Korean files being inside the en/ directory beside the English ones.

In your mkdocs.yml file you have fallback_to_default: true, here is the documentation: https://ultrabug.github.io/mkdocs-static-i18n/setup/controlling-your-builds/#fallbacking-to-default If there is no file under the same path in the en/ docs directory it will copy it from ko/, as you're using the Korean letters for Korean version and English letters for English version, then the path is not found and it's detected as a lacking file in the en/ version compared to the Korean one and then it copies the files over.

So try setting fallback_to_default: false @snowCho and it should help.

snowCho commented 5 months ago

@kamilkrzyskow oh! The fallback_to_default: false option solved the problem.

Thank you so much for your kind help!

kamilkrzyskow commented 5 months ago

No problem, sorry it took so long, and for confusing you ๐Ÿ˜“I should know what you meant from the first post, but I started off with an assumption that there is some error in the code after my changes instead of the mkdocs.yml config... If the issue is resolved please close the discussion โœŒ๏ธ