Fail to translate a page when it includes table block or 100+ children under a block

pavzagor commented 1 year ago

Opening a new issue so that the commit comment doesn't get lost.

seratch commented 1 year ago

@pavzagor Thanks for reporting this! I may be able to secure some time to check the issue on the weekend.

If you can share a bit more details of the page such as the page retrieval results etc. may help me identify the issue more easily. Also, having --debug option when running the command provides more detailed logs. If you can share the outputs, it can be also helpful too.

seratch commented 1 year ago

I've checked this issue and had to skip the patterns below:

When there is a table block on a page, the translator skips the block and displays a disclaimer block instead
When 100+ children in a block, the 100th, and following blocks are removed in the translated page displays a disclaimer block as 100th

seratch commented 1 year ago

I just released v0.0.4, which resolves the issue. Please upgrade the version and try it out

RuslanCheboksarov commented 1 year ago

Hello, the issue still remains

Node.js v19.0.1

pavzagor-Macbook-pro-14:~ pavelzagorodnikh$ notion-translator -u https://www.notion.so/kodland/Magic-of-code-with-Scratch-M4L4-20-DEBUG-NOTION-TRANSLATE-CLI-f12559d207904da09777ed548f6abf6e -f ru -t en-us

Wait a minute! Now translating the following Notion page:
https://www.notion.so/kodland/Magic-of-code-with-Scratch-M4L4-20-DEBUG-NOTION-TRANSLATE-CLI-f12559d207904da09777ed548f6abf6e

(this may take some time) ........................................./opt/homebrew/lib/node_modules/notion-translator/index.js:380
  const originalTitle = originalPage.properties.title.title[0];
                                                      ^

TypeError: Cannot read properties of undefined (reading 'title')
    at createNewPageForTranslation (/opt/homebrew/lib/node_modules/notion-translator/index.js:380:55)
    at /opt/homebrew/lib/node_modules/notion-translator/index.js:426:25
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

RuslanCheboksarov commented 1 year ago

Just as validation error:

(this may take some time) ......................................../opt/homebrew/lib/node_modules/notion-translator/node_modules/@notionhq/client/build/src/errors.js:162
return new APIResponseError({
^
APIResponseError: body failed validation: body.children[1].column_list.children[0].column.children should be defined, instead was undefined.
at buildRequestError (/opt/homebrew/lib/node_modules/notion-translator/node_modules/@notionhq/client/build/src/errors.js:162:16)
at Client.request (/opt/homebrew/lib/node_modules/notion-translator/node_modules/@notionhq/client/build/src/Client.js:304:54)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /opt/homebrew/lib/node_modules/notion-translator/index.js:436:26 {
code: 'validation_error',
status: 400,
headers: Headers {
[Symbol(map)]: [Object: null prototype] {
date: [ 'Fri, 18 Nov 2022 13:04:49 GMT' ],
'content-type': [ 'application/json; charset=utf-8' ],
'content-length': [ '194' ],
connection: [ 'close' ],
'set-cookie': [
'notion_browser_id=7c476af3-47e8-4590-b759-0d84ac01d42c; [Domain=www.notion.so](http://domain/%3Dwww.notion.so/); Path=/; Expires=Sat, 18 Nov 2023 13:04:49 GMT; Secure',
'notion_check_cookie_consent=false; [Domain=www.notion.so](http://domain/%3Dwww.notion.so/); Path=/; Expires=Sat, 19 Nov 2022 13:04:49 GMT; Secure'
],
'content-security-policy': [
"script-src 'self' 'unsafe-inline' 'unsafe-eval' https://gist.github.com/ https://apis.google.com/ https://www.google.com/ https://www.gstatic.com/ https://cdn.amplitude.com/ https://api.amplitude.com/ https://hkfxbbdzib.notion.so/ https://widget.intercom.io/ https://js.intercomcdn.com/ https://static.zdassets.com/ https://api.smooch.io/\t https://logs-01.loggly.com/ https://http-inputs-notion.splunkcloud.com/ https://cdn.segment.com/ https://analytics.pgncs.notion.so/ https://o324374.ingest.sentry.io/ https://checkout.stripe.com/ https://js.stripe.com/ https://embed.typeform.com/ https://admin.typeform.com/ https://public.profitwell.com/ [js.sentry-cdn.com](http://js.sentry-cdn.com/) https://js.chilipiper.com/ https://platform.twitter.com/ https://cdn.syndication.twimg.com/ https://accounts.google.com/ https://www.googletagmanager.com/ https://www.googleadservices.com/ https://googleads.g.doubleclick.net/ https://api-v2.mutinyhq.io/ https://client-registry.mutinycdn.com/ https://client.mutinycdn.com/ https://user-data.mutinycdn.com/ https://metadata-static-files.sfo2.cdn.digitaloceanspaces.com/ https://platformapi.metadata.io/ https://cdn01.boxcdn.net/;connect-src 'self' https://msgstore.www.notion.so/ wss://msgstore.www.notion.so ws://localhost:* ws://127.0.0.1:* https://notion-emojis.s3-us-west-2.amazonaws.com/ https://s3-us-west-2.amazonaws.com/ https://s3.us-west-2.amazonaws.com/ https://notion-production-snapshots-2.s3.us-west-2.amazonaws.com/ https://cdn.amplitude.com/ https://api.amplitude.com/ https://hkfxbbdzib.notion.so/ https://www.notion.so/ https://api.embed.ly/ https://js.intercomcdn.com/ https://api-iam.intercom.io/ https://uploads.intercomcdn.com/ wss://nexus-websocket-a.intercom.io https://ekr.zdassets.com/ https://ekr.zendesk.com/\t https://makenotion.zendesk.com/\t https://api.smooch.io/\t wss://api.smooch.io\t https://logs-01.loggly.com/ https://http-inputs-notion.splunkcloud.com/ https://cdn.segment.com/ https://api.segment.io/ https://analytics.pgncs.notion.so/ https://api.pgncs.notion.so/ https://o324374.ingest.sentry.io/ https://checkout.stripe.com/ https://js.stripe.com/ https://cdn.contentful.com/ https://preview.contentful.com/ https://images.ctfassets.net/ https://www2.profitwell.com/ https://tracking.chilipiper.com/ https://api.chilipiper.com/ https://api.unsplash.com/ https://boards-api.greenhouse.io/ https://accounts.google.com/ https://oauth2.googleapis.com/ https://www.googletagmanager.com/ https://analytics.google.com/ https://www.googleadservices.com/ https://googleads.g.doubleclick.net/ https://region1.google-analytics.com/ https://region1.analytics.google.com/ https://www.google-analytics.com/ https://api-v2.mutinyhq.io/ https://client-registry.mutinycdn.com/ https://client.mutinycdn.com/ https://user-data.mutinycdn.com/ https://metadata-static-files.sfo2.cdn.digitaloceanspaces.com/ https://platformapi.metadata.io/ https://api.statuspage.io/ https://pgncd.notion.so/ https://api.statsig.com/ https://statsigapi.net/ https://exp.notion.so/ https://file.notion.so/ https://api.box.com/;font-src 'self' data: https://cdnjs.cloudflare.com/ https://js.intercomcdn.com/ https://cdn01.boxcdn.net/;img-src 'self' data: blob: https: https://platform.twitter.com/ https://syndication.twitter.com/ https://pbs.twimg.com/ https://ton.twimg.com/ https://region1.google-analytics.com/ https://region1.analytics.google.com/;style-src 'self' 'unsafe-inline' https://cdnjs.cloudflare.com/ https://github.githubassets.com/ https://js.chilipiper.com/ https://platform.twitter.com/ https://ton.twimg.com/ https://accounts.google.com/ https://cdn01.boxcdn.net/;frame-src https: http: https://accounts.google.com/;media-src https: http: https://file.notion.so/"
],
'x-dns-prefetch-control': [ 'off' ],
'x-frame-options': [ 'SAMEORIGIN' ],
'strict-transport-security': [ 'max-age=5184000; includeSubDomains' ],
'x-download-options': [ 'noopen' ],
'x-content-type-options': [ 'nosniff' ],
'x-permitted-cross-domain-policies': [ 'none' ],
'referrer-policy': [ 'strict-origin-when-cross-origin' ],
'x-xss-protection': [ '0' ],
etag: [ 'W/"c2-k2Z2DyiSbclk4VK7K7CuQP/QXoA"' ],
vary: [ 'Accept-Encoding' ],
'cf-cache-status': [ 'DYNAMIC' ],
server: [ 'cloudflare' ],
'cf-ray': [ '76c0ea03484ab6ea-KIV' ]
}
},
body: '{"object":"error","status":400,"code":"validation_error","message":"body failed validation: body.children[1].column_list.children[0].column.children should be defined, instead was undefined."}'
}
pavzagor-Macbook-pro-14:~ pavelzagorodnikh$ notion-translator -u https://www.notion.so/kodland/Magic-of-code-with-Scratch-M4L4-20-DEBUG-NOTION-TRANSLATE-CLI-f12559d207904da09777ed548f6abf6e -f ru -t en-us

Wait a minute! Now translating the following Notion page:
https://www.notion.so/kodland/Magic-of-code-with-Scratch-M4L4-20-DEBUG-NOTION-TRANSLATE-CLI-f12559d207904da09777ed548f6abf6e

seratch commented 1 year ago

@pavzagor I've copied your page but I could not reproduce these issues. I cannot think of the situation where title property is undefined as it is required to build a page. Also, I am not sure about the cause of child_list issue.

I am so happy to see you are relying on this tool! but unfortunately I cannot use my time a lot for this project on the weekend. If you can provide an easier way to reproduce the issue such as automated test code, fixing all the issues you see would be much easier for me.

seratch commented 1 year ago

Applied a workaround for the title issue and released v0.0.5: https://github.com/seratch/notion-translator/commit/b611f4ee26a1ceee2827d53be25d874c2970d945

I am not yet sure what the best solution for the column_list issue is.

seratch / notion-translator

Fail to translate a page when it includes table block or 100+ children under a block #4