Closed xplosionmind closed 2 years ago
Actually, it appears that I am having a problem with filters in general… replace
is supported by LiquidJS, yet I get this error, too:
[11ty] Problem writing Eleventy templates: (more in DEBUG output)
[11ty] > Having trouble rendering liquid template ./pages/whole-jam.html
`TemplateContentRenderError` was thrown
[11ty] > Cannot read properties of undefined (reading 'replace'), file:./pages/whole-jam.html, line:23, col:9
`RenderError` was thrown
[11ty] > Cannot read properties of undefined (reading 'replace')
`TypeError` was thrown:
[11ty] TypeError: Cannot read properties of undefined (reading 'replace')
at Object.stripHtml (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:3712:14)
at Filter.render (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:2405:26)
at Value.<anonymous> (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:2441:49)
at step (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:87:23)
at Object.next (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:68:53)
at reduce (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1044:25)
at Object.then (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1012:43)
at reduce (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1051:40)
at toThenable (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1039:16)
at reduce (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1051:16)
[11ty] Wrote 0 files in 0.67 seconds (v1.0.0)
Hhmm, that's curious... Not sure why Eleventy wouldn't be finding your .eleventy.js config file. Maybe try throwing some console.log()
statements in the config file and verify that you see the debug output in your console. Or possibly a typo in .eleventy.js filename and it is using a default config.
Do you see this on a brand new Eleventy project as well? Or is this specific to your big tommi.space blog? Might be easier to troubleshoot if you start a new "hello world" style Eleventy blog and push it to GitHub and we can take a look. Or maybe try cloning my small https://github.com/pdehaan/11ty-2229 repo and seeing if you can build that locally.
Curious if you have a globally installed @11ty/eleventy module, or how you're building the site (npx or a script in package.json).
By adding console.log();
to my .eleventy.js
file, nothing changes in the output. I am quite confident that there are no typos, though. Nevertheless, by running DEBUG=Eleventy* eleventy --dryrun
I get this infinite log that you might find insightful.
The problem is specific to my tommi.space blog. About that: I uploaded the source code for the website compiled with Eleventy (branch 11ty
) on here… I am still using Jekyll for production (main
branch). I cloned and run the repository you kindly created in order to test this issue, and everything runs smoothly there.
I both tried the local and global installations, the issue is the same. Currently using the global.
Transitioning to Eleventy is sooo hard given the original complexity of my website with Jekyll; nevertheless, doing a step by step transition with one file at a time is even harder, since all of layouts/includes/etc. are heavily intertwined. Please feel free to open a PR on https://github.com/xplosionmind/tommi.space 11ty
branch.
OK, managed to clone your repo/branch, but the _data directory is empty and throwing errors in the build. How do I fetch the git submodules? I tried a few stackoverflow tips, but seem to always get the following output:
Fetching submodule _data
Already up to date.
Cloning into '/private/tmp/tommi.space/_notes/PISE'...
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:xplosionmind/pise-notes.git' into submodule path '/private/tmp/tommi.space/_notes/PISE' failed
Failed to clone '_notes/PISE'. Retry scheduled
Cloning into '/private/tmp/tommi.space/_notes/PISE'...
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:xplosionmind/pise-notes.git' into submodule path '/private/tmp/tommi.space/_notes/PISE' failed
Failed to clone '_notes/PISE' a second time, aborting
And my _data directory still seems to be empty:
tree -a _data _notes
_data/
└── .git/
_notes/
├── PISE/
└── public/
├── A Rainy Day in New York.md
├── A script will do.md
├── Aaron Swartz.md
├── ...
├── Zig.md
├── dotfiles.md
├── git.md
├── post-comprensione.md
└── reMarkable.md
2 directories, 232 files
Alright, I managed to just clone the data git submodule manually into the _data directory.
After some debugging, I managed to get the site building w/ Eleventy v1.0.0:
npm run build -- --quiet
> tommi.space@1.0.0 build
> eleventy "--quiet"
Eleventy config in the house
[11ty] Wrote 348 files in 0.35 seconds (1.0ms each, v1.0.0)
I expect this to be ugly, but here's a git diff
of my changes. I'll see if I can split up the results to give a bit more context.
git diff -- ':!package-lock.json' | pbcopy
{% comment %}FIX{{ note.description | markdownify | strip_html | truncatewords: 30 }}{% endcomment %}
+ {% comment %} {% if note.description %} + {% else %}{{ note.excerpt | strip_html | truncatewords: 30 }}
- {% endif %} + {% endif %} {% endcomment %}Oh, and another thing that I think is causing issues (and might be a Jekyll-ism) was the use of site.*
.
The git data
submodule has the following https://github.com/xplosionmind/data/blob/main/site.json file which will be added to the _data/ folder as a global data file (and can then be referenced as {{ site.title }}
, etc):
{
"title": "tommi.space",
"email": "surfing@tommi.space",
"description": "A virtual representation of the mess inside Tommi’s mind",
"baseurl": "",
"url": "https://tommi.space"
}
But with a bit of grep, this should be a semi-accurate list of references to site.
variables in your templates/content (you can omit the -h
flag if you want to display filenames for each reference):
git grep -h -oP "\b(site\..*?)\s" | sort | uniq
site.author.name
site.baseurl
site.data.accounts
site.data.biografie
site.data.books-wishlist
site.data.deleted
site.data.giri
site.data.internet-awesomeness
site.data.library
site.data.lost
site.data.parole
site.data.people
site.data.places
site.data.quotes
site.data.stuff
site.data.tags
site.data.tommi-uguale
site.data.tutto
site.data.watchlist
site.data.watchlog
site.data.webrings
site.email
site.filinge
site.git_repository
site.github_username
site.image
site.images
site.last_modified_at
site.logos
site.notes
site.pages
site.podcast.author
site.podcast.block
site.podcast.complete
site.podcast.description
site.podcast.email
site.podcast.link
site.podcast.logo
site.podcast.title
site.podcast.type
site.poetry
site.post-images
site.posts
site.sconnesso
site.title
site.url
Again, not all of those will be valid. They might be used in Markdown files like site.poetry
as a string, and not used in a Liquid tag or filter.
I didn't verify any of those on the website, but my guess is that those would evaluate to undefined
and be causing some of the issues we were seeing above.
I'm guessing the site.data.*
references might be referring to the .CSV files in the _data/ directory. But I don't think Eleventy will do that automatically for you, so you might need to find an alternative way of converting those CSVs into JSON. I haven't tried CSV files specifically, but you might be able to use https://www.11ty.dev/docs/data-custom/. But even then, you'd refer to the files as just {{ quotes }}
and not {{ site.data.quotes }}
(which would expect either a .data.quotes
property in the site.json file, or possibly a _data/site/data/quotes.csv file):
tree _data/
_data/
├── accounts.csv
├── apps.csv
├── biografie.csv
├── books-wishlist.csv
├── giri.csv
├── internet-awesomeness.csv
├── library.csv
├── lost.csv
├── parole.csv
├── people.csv
├── places.csv
├── quotes.csv
├── readslog.csv
├── site.json
├── stuff.csv
├── tags.csv
├── timeline.csv
├── tommi-uguale.csv
├── tutto.yml
├── tutto_old.csv
├── watchlist.csv
├── watchlog.csv
└── webrings.csv
0 directories, 23 files
Alternatively, you could maybe change the site.json file to site.js and then just use module.exports = {}
syntax and write some code to load the CSV files, parse them, and export them as JSON. I've used https://csv.js.org/parse/api/sync/ in the past. Not impossible to do, but maybe challenging since those files are in a git submodule and I don't know what else uses the data.
But the syntax would probably look something like:
// _data/site.js
module.exports = {
"title": "tommi.space",
"email": "surfing@tommi.space",
"description": "A virtual representation of the mess inside Tommi’s mind",
"baseurl": "",
"url": "https://tommi.space/",
"data": {
"tutto": {...},
"watchlist": {...}
}
};
Alright, sorry, the suspense was bugging me. I verified that you can use .CSV as a valid data file by installing a library like csv-parse (npm i csv-parse -D), and adding this to my .eleventy.js config file:
+// npm i csv-parse -D
+const csvParse = require('csv-parse/sync').parse;
module.exports = function(eleventyConfig) {
+ eleventyConfig.addDataExtension("csv", contents => csvParse(contents, {columns: true, skip_empty_lines: true}));
eleventyConfig.addFilter("inspect", require("node:util").inspect);
...
};
Then I created a test ./pages/quotes.html file with the following:
---
title: Quotes y'all!
---
<h1>{{ title }}</h1>
{% comment %} {{ quotes | where: "lang", "it" | inspect }} {% endcomment %}
{%- assign itQuotes = quotes | where: "lang", "it" -%}
{%- for quote in quotes %}
<p>{{ quote.author }}: "{{ quote.quote }}"</p>
{%- endfor %}
<h1>Quotes y'all!</h1>
<p>Matto: "Con Tommi ho una relazione molto più solida che con Roberta e lei lo sa"</p>
<p>Carlo Lascialfari: "…nelle dinamiche di una vita l’unica cosa che conta meno di un esame fatto male è un esame fatto bene"</p>
<p>Michele Apicella: "Non faccio mai conoscere fra loro le persone a cui tengo"</p>
...
<p>: ""</p>
Not sure why there are a couple "empty" quotes in there, but I'm guessing it's due to the ,,,,,,,,,
entries in quotes.csv (L90+L93). Although it's unclear to me why that would have passed the where: "lang", "it"
filter. 🤷
The CSV parser also flagged at least 2 errors in your CSV files where there was more data than columns, so I either deleted a trailing comma, or added quotes around some names to make the number of columns match the number of headers. But because of the git submodule and my weird git clone thing, git isn't showing which files I needed to modify, sorry. (I think one of them might have been _data/books-wishlist.csv, the other was possibly _data/quotes.csv.)
UPDATE: Amazingly, it also seems to just work(tm) for global data files w/ hyphens in the name (although that usually famously breaks hard elsewhere since "books-wishlist" isn't a valid JavaScript variable name:
---
title: Books, y'all!
---
{%- for book in books-wishlist %}
<p>{{ book.author }}: {{ book.title }}</p>
{%- endfor %}
<p>Beckett: En attandant Godot</p>
<p>Borelli: Metodi di manipolazione</p>
<p>Bückler: Non esistono piccole storie</p>
...
<p>: </p>
And looks like another empty book due to ,,
in the books-wishlist.csv file. Inspecting the converted CSV=>JSON, it looks like that record converts to:
{ author: '', title: '', date: '' },
Not sure how best to solve that. Easiest is to fix the input data and remove those invalid lines. Less easy is a custom filter which filters out any results that have empty author
and/or title
and/or date
fields.
First of all, a huge thank you for your help in debugging everything!
Unfortunately, I am still having many issues with the configuration:
I don't think this matters right now but it was confusing to have a
return
in the middle of the function. Nothing after the return would be processed (if it wasn't commented out anyways) but that might explain other errors about Eleventy/LiquidJS not being able to find areverse
ormarkdownify
filter, etc.
.eleventy.js
as you suggested, and putting the return function at the bottom, I still get Liquid filters errors (see the following points: intertwined issues)!Something somewhere really didn't like
url_encode
as a filter. Removed it and moved on.
url_encode
should be a valid filter! What is the problem? Should I open another issue? (still related to #2229, though)replace
), that is not recognized:eleventy
[11ty] Problem writing Eleventy templates: (more in DEBUG output)
[11ty] > Having trouble rendering liquid template ./pages/whole-jam.html
`TemplateContentRenderError` was thrown
[11ty] > Cannot read properties of undefined (reading 'replace'), file:./pages/whole-jam.html, line:23, col:9
`RenderError` was thrown
[11ty] > Cannot read properties of undefined (reading 'replace')
`TypeError` was thrown:
[11ty] TypeError: Cannot read properties of undefined (reading 'replace')
at Object.stripHtml (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:3712:14)
at Filter.render (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:2405:26)
at Value.<anonymous> (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:2441:49)
at step (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:87:23)
at Object.next (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:68:53)
at reduce (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1044:25)
at Object.then (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1012:43)
at reduce (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1051:40)
at toThenable (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1039:16)
at reduce (/usr/local/lib/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:1051:16)
[11ty] Wrote 0 files in 0.93 seconds (v1.0.0)
- Even by changing
.eleventy.js
as you suggested, and putting the return function at the bottom, I still get Liquid filters errors (see the following points: intertwined issues)!
Yes, moving the return
wasn't solving everything but was explaining why you might have been seeing other errors with your custom filters. But hard to tell with the commented out code. It was more of a potential gotcha.
- According to LiquidJS documentation,
url_encode
should be a valid filter! What is the problem? Should I open another issue? (still related to Unknown Liquid filters and variables block build even if strict is off #2229, though)
No, we don't need any more bugs. LiquidJS and Eleventy seem to both be working fine, this seems to be specific to your blog/config.
- Actually, it is quite useless for me to continue going through your remarks, since my errors at build time are still caused by a (reportedly valid) Liquid filter (
replace
), that is not recognized:
Well, I wouldn't say useless, but definitely some of them were just placeholders for future CSV stuff. So did you go through all of the diffs and your site still isn't working locally for you? I can reproduce the liquid errors with the following simplified example:
---
title: OK, LET'S GO
---
<h1>{{ title }}</h1>
{{ undefined_value | url_encode }}
Gives me the following build error:
> 11ty-liquid-undefined@1.0.0 build > eleventy [11ty] Problem writing Eleventy templates: (more in DEBUG output) [11ty] > Having trouble rendering liquid template ./src/index.liquid `TemplateContentRenderError` was thrown [11ty] > Cannot read properties of undefined (reading 'split'), file:./src/index.liquid, line:4, col:1 `RenderError` was thrown [11ty] > Cannot read properties of undefined (reading 'split') > `TypeError` was thrown: [11ty] TypeError: Cannot read properties of undefined (reading 'split') at Object.urlEncode (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:3744:41) at Filter.render (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:2407:26) at Value.<anonymous> (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:2443:49) at step (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:87:23) at Object.next (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:68:53) at reduce (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:1046:25) at Object.then (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:1014:43) at reduce (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:1053:40) at toThenable (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:1041:16) at reduce (/private/tmp/11ty-liquid-undefined/node_modules/liquidjs/dist/liquid.node.cjs.js:1053:16) [11ty] Wrote 0 files in 0.02 seconds (v1.0.0) The terminal process "/bin/zsh '-c', 'npm run build'" terminated with exit code: 1.
Basically that is doing {{ undefined | url_encode }}
and failing. So I don't think LiquidJS filters always handle undefined
values (which is fine).
Re: Cannot read properties of undefined (reading 'split')
:
I'm fairly confident that this error is saying it's trying to call .split()
on an undefined
value. It isn't saying that the split
or in your case replace
filters can't be found. I think Eleventy/LiquidJS is finding the filters, but the inputs are just all undefined
.
So before going any further, I'd try and find out why all these values in your site are undefined
, which we kind of covered above with the invalid references to stuff like site.data.library
which is going to be undefined
using your current config (by fixing the site.data.*
references and/or tweaking the CSV data extension stuff).
But yes, you have a relatively large (350 page) existing site written for another blog engine, so there is definitely an added learning curve to converting things from Jekyll to Eleventy since they both support different features and you'll either need to update your code/data/templates, or write a bunch of custom filters/parsers yourself. I don't think we'll find a single setting or config that will get this to work without a lot of tweaking.
You have been very, very, very helpful.
I went through all of your points and I managed to make the site run locally. The problem was indeed the “undefined” nature of the elements to be filtered, rather than the filter itself. I am now proceeding in the debugging by resolving problems that are not build-blocking.
Since as you mentioned more than once that this issue was about my specific website rather than an Eleventy one, I will close this issue with this comment.
Again, thank you very much, @pdehaan!
Yeah, I'll add that I got the build working locally BUT I never actually verified the output of the site. I just say "348 files built" and walked away. So it's very much likely that the pages werent rendering as expected due to Eleventy not being able to find expected data or collections or whatever.
I might file an upstream LiquidJS about {{ undefined | url_encode }}
and see if they have opinions. I could be convinced on the behavior either way. On one hand it's nice that the build failed on bad inputs. But curious if there's another flag or something we could set to temporarily ignore those errors.
UPDATE: Done, https://github.com/harttle/liquidjs/issues/479.
But yeah, I'd be interested in your results after you're done converting your site.
I haven't used Jekyll in a super long time, so curious what issues you'll encounter while migrating (apart from the lack of the special Jekyll filters — although I expect most of those might be easy enough to recreate yourself; except any that end with *_exp
).
Describe the bug I am creating in my
.eleventy.js
some custom filters, yet they are not being recognized.To Reproduce Steps to reproduce the behavior:
.eleventy.js
:module.exports = function (eleventyConfig) { eleventyConfig.addLiquidFilter('reverse', (collection) => { const arr = [...collection]; return arr.reverse(); }); eleventyConfig.addFilter('markdownify', (str) => { return markdownItRenderer.renderInline(str) }) };
[11ty] Problem writing Eleventy templates: (more in DEBUG output) [11ty] > Having trouble rendering liquid template ./pages/search.html
TemplateContentRenderError
was thrown [11ty] > undefined filter: markdownify, file:./pages/search.html, line:8, col:13, file:./pages/search.html, line:3, col:18ParseError
was thrown [11ty] > undefined filter: markdownify, file:./pages/search.html, line:8, col:13ParseError
was thrown [11ty] > undefined filter: markdownifyAssertionError
was thrown: [11ty] AssertionError: undefined filter: markdownify at new AssertionError (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:664:28) at assert (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:822:15) at FilterMap.get (/Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:4254:9) at /Users/tommi/tommi.space/node_modules/@11ty/eleventy/node_modules/liquidjs/dist/liquid.node.cjs.js:2420:52 at Array.map (