search

danielfrg / pelican-jupyter

Pelican plugin for blogging with Jupyter/IPython Notebooks
Apache License 2.0
423 stars 105 forks source link

Plugin doesn't work with nbconvert >= 6.x #126

Open andreagrandi opened 3 years ago

andreagrandi commented 3 years ago

Hello,

I think latest Pelican version may have introduced some breaking changes for plugin (see this post here https://blog.getpelican.com/pelican-4.5-released.html ). If I upgrade from pelican==4.2.0 to pelican==4.5.0 I get this error when I build my website containing an .ipydb file:

ERROR: Could not process 2018/1-PimaIndiansDiabetes.ipynb
  | basic

I haven't been able to get more information. It just works fine with 4.2.0 but it doesn't work with 4.5.0

Thanks

andreagrandi commented 3 years ago

I did a few more tests and there seems to be an incompatibility with nbconvert >= 6.

I had accidentally upgraded that package and the plugin couldn't work anymore.

As soon as I installed back the latest 5.x available, the plugin worked (with Pelican 4.5.0).

I think we should pin the version in requirements to be >=5,<6 (or whatever the syntax is, I don't remember it by memory).

This as a short term fix of course. For the long term it would be interesting to understand what changed in 6.x

Cheers

danielfrg commented 3 years ago

Thanks for the report @andreagrandi a lot of packages are broken right now because of nbconvert 6.0.0, so yeah we should totally pin this.

I am personally not using this plugin anymore so I wont be working on this in the near future. If you want to take look at it please do so.

andreagrandi commented 3 years ago

Thanks for your quick reply Daniel! I've done a tiny PR to pin the version https://github.com/danielfrg/pelican-jupyter/pull/127 I'm not sure if you are still available to merge and release it by the way. In case you think you are not able to maintain this anymore, would it be possible to become maintainer for the PYPI package here https://pypi.org/project/pelican-jupyter/ ? This way I could keep developing and fixing the package on my own repository but this would not disrupt future releases. Cheers

andreagrandi commented 3 years ago

Actually I'm struggling to make my PR work, since there are a few isort problems which are failing on CI but I can't reproduce on my local machine, so don't bother yet.

danielfrg commented 3 years ago

@andreagrandi thanks for the offer, i am happy to add you as maintainer here and on PyPI! I will do it on GitHub now. Let me know your PyPI username and I will also add you there.

isort recently broke the CI linting on some of my projects, I personally just removed that from the CI as its not critical but up to you to just disable it.

andreagrandi commented 3 years ago

I'm andreagrandi on PyPI too.

About isort: where did you remove it from exactly? It seems that it's still running when I push the code, that's why the build is failing. I will try to use exactly the same version (or pin its version to the same I use locally), because I haven't been able to reproduce the same problem locally on my machine.

danielfrg commented 3 years ago

Added you to pypi.

I removed it from the github actions until i see why are they failing.

dclong commented 3 years ago

@danielfrg Are you using another better alternative or do you simply give up using Jupyter for Pelican?

danielfrg commented 3 years ago

I am no longer using pelican for any websites. I use mkdocs for docs so I wrote: https://github.com/danielfrg/mkdocs-jupyter And for my website I just use hugo and a similar script.

flothesof commented 3 years ago

I believe I have the same problem: rendering of notebooks have suddenly failed when my automated build updated to the latest version.

Using the built-in option for specifying a template, I managed to find a workaround.

LIQUID_CONFIGS = (("IPYNB_EXPORT_TEMPLATE", "base", ""),)

If my reasoning is correct, the template called "basic" has been renamed to "base" by nbconvert, which causes the trouble of the notebooks not rendering anymore.

I therefore conclude that changing this line might solve this problem: https://github.com/danielfrg/pelican-jupyter/blob/ce29872df1999a1ea3a373ca9e9d053292d55bbf/pelican_jupyter/core.py#L89

Judging by bug reports over at nbconvert, it seems that quite a lot of template-related things have been broken by release 6.0:

https://github.com/jupyter/nbconvert/issues/1451#issuecomment-709391371

shanewilkins commented 3 years ago

Confirm that the upgrade to nbconvert 6.0 is the issue. I downgraded back to 5.6 and the error went away. I haven't been able to test whether @flothesof 's fix will work.

johnmathews commented 3 years ago

As well as downgrading nbconvert to 5.6.0, I also needed to downgrade notebook to 5.6.0. The suggestion from @flothesof didn't make any difference for me.

flothesof commented 3 years ago

@johnmathews prompted by your finding I tried again and was able to replicate my workaround from (almost) scratch. Version info:

Here's what I had to put into my pelicanconf.py (only relevant lines included):

from pelican_jupyter import liquid as nb_liquid # import pelican-jupyter
...
PLUGINS = ['liquid_tags.img', nb_liquid, 'representative_image']
...
# configuring notebook integration
LIQUID_CONFIGS = (("IPYNB_FIX_CSS", "False", ""), 
                  ("IPYNB_SKIP_CSS", "False", ""), 
                  ("IPYNB_EXPORT_TEMPLATE", "base", ""),)

Maybe this works better for you?

shanewilkins commented 3 years ago

@flothesof This seems to have done the trick for me. All of the plotly plots in my notebooks have disappeared, but that's a separate issue.

AndreasThinks commented 3 years ago

Glad I'm not the only one! It looks like it's an issue pulling out the Jinja2 template or similar...May have to explore alternative solutions to good Python/Jupyter static pages! Have been pondering swapping to Hugo for awhile, may be the time to make the jump.

alex000kim commented 3 years ago

downgrading to python 3.6, nbconvert=5.6.0, notebook=5.6.0 helps, but then plotly plots stop rendering...

davidwagner commented 3 years ago

Changing basic.tpl to classic/base.html.j2 in plugins/ipynb/core.py seems to also fix this, without needing to downgrade nbconvert. I have a somewhat older version of pelican-jupyter though so this might need to be translated to the latest version.

I wonder if something like

template_file = 'classic/base.html' if int(nbconvert.__version__.partition(".")[0]) >= 6 else 'basic'

might work.

canyon289 commented 2 years ago

Im having this same issue as well. Tagging myself onto the discussion here!

magsol commented 2 years ago

Can confirm this is still an issue. Here is my relevant environment:

Can also confirm that the fix posted by @flothesof still works, at least in terms of the LIQUID_CONFIGS.

I did have a slight modification I had to make, as it appeared that having both pelican-jupyter and pelican-liquid-tags register the same notebook tag was causing problems: I could either get Jupyter notebooks to render, OR any of the other liquid tags (images, videos, code, etc). So I moved the pelican-jupyter liquid.py file over to the pelican-liquid-tags folder, renamed it notebook.py, and slightly modified the imports.

After this, my pelicanconf.py looked like this:

from pelican.plugins import liquid_tags
PLUGINS = [
        liquid_tags,                # regular ol' liquid tags
]
LIQUID_TAGS = ["img", "include_code", "gram", "video", "youtube", "notebook"]
LIQUID_CONFIGS = (("IPYNB_FIX_CSS", "False", ""), 
                  ("IPYNB_SKIP_CSS", "False", ""), 
                  ("IPYNB_EXPORT_TEMPLATE", "base", ""),)

Seems to work. It's not pretty but I couldn't find a way to have the two plugins co-exist.

ozak commented 2 years ago

Same error. After following the advice above I am getting the following error

could not find information about 'title'

But I have both the nbdata and ipynb-meta files for each notebook in the same folder as the notebook, and it used to work with previous versions. Any ideas would be appreciated.

flothesof commented 2 years ago

I do get this error for some files that are in my repository, but this is never a problem in my case. Did it not generate any pages with your setup?

Le mer. 19 janv. 2022 à 03:39, Ömer Özak @.***> a écrit :

Same error. After following the advice above I am getting the following error

could not find information about 'title'

But I have both the nbdata and ipynb-meta files for each notebook in the same folder as the notebook, and it used to work with previous versions. Any ideas would be appreciated.

— Reply to this email directly, view it on GitHub https://github.com/danielfrg/pelican-jupyter/issues/126#issuecomment-1016024188, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASAKXLN3VNJ3ZKBG2CAKSTUWYP5RANCNFSM4SCXTLLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

ozak commented 2 years ago

No, it fails completely and does not generate anything. I also get an additional error

CRITICAL UndefinedError: 'gettext' is undefined

which I have no idea what is causing it.

flothesof commented 1 year ago

@danielfrg Do you have any ideas how we could get out of this situation that the current version of the plugin is not working as expected anymore? Would you be willing to transfer maintainership to someone else to get the plugin to work with current versions of nbconvert since you don’t seem to have time to work on this? Regards, Florian

andreagrandi commented 1 year ago

@danielfrg Do you have any ideas how we could get out of this situation that the current version of the plugin is not working as expected anymore? Would you be willing to transfer maintainership to someone else to get the plugin to work with current versions of nbconvert since you don’t seem to have time to work on this? Regards, Florian

I also "maintain" (I did a release but stuff in real life happened and couldn't do anymore) the package on PyPI https://pypi.org/project/pelican-jupyter/ but I've no idea how to fix this. Plus I haven't used the library in ages (I haven't since then written anything which requires this package to be working).

flothesof commented 1 year ago

@andreagrandi thank you for coming back on this topic Actually, support for notebooks also works with the liquid-tags plugin and seems to be functional. This could help some people stuck here: https://github.com/pelican-plugins/liquid-tags#ipython-notebooks