Closed haydenbleasel closed 1 month ago
This is strange, because their is no timeout for the transform function. Can you test, if you get the same issue when you use the cli instead of the next adapter?
You should definitely cache the blur generation, to speed up rebuilds: https://www.content-collections.dev/docs/transform#caching
@sdorra Seems to work fine when using the CLI. Maybe it's a race condition specifically with the Next.js adapter?
Thanks for the info on caching! Combining caching with a prebuild
script of pnpm content-collections build
does the trick. I guess it's an issue specifically with the Next.js adapter.
@haydenbleasel I've tried to reproduce the issue by using a timeout:
import { defineCollection, defineConfig } from "@content-collections/core";
const characters = defineCollection({
name: "characters",
directory: "characters",
include: "*.md",
schema: (z) => ({
name: z.string().min(1),
origin: z.string().min(1),
species: z.string().min(1),
source: z.string().min(1).url(),
}),
transform: async (data) => {
console.log(`transforming ${data.name} ...`)
for (let i=0; i<60; i++) {
console.log(`Waiting for 1 second... (${i+1}/60)`);
await new Promise((resolve) => setTimeout(resolve, 1000));
}
return data;
},
});
export default defineConfig({
collections: [characters],
});
I've tested with 5 Markdown files, so content collections run for 5 minutes and I've got no timeout. Can you help me reproduce the issue? Is the repository that requires the prebuild step public?
I'm closing the issue now because I can't reproduce the problem. Please reopen the issue if you can help me reproduce the issue.
I have the following collection:
The longest running part of it is of course importing the referenced image and creating a low-quality image placeholder. This takes maybe a minute with the amount of images I'm passing it.
Before the transforms start,
.content-collections/generated/index.js
is created with the following contents:allBlogs.js
is usually generated shortly after, once all the content is done transforming. However, if thetransform
function takes too long, it times out andallBlogs.js
isn't generated, leading to the following build log:Running on Next.js 14.2.5 on Node.js v20.12.2.