Multilingual gatsby-source-drupal translatable media returning null file

apmsooner commented 3 years ago

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
[X] This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

Files are non-translatable entities in Drupal but media entities are and all languages should return the file. If I set a media type in the translatableEntities array, with example of 2 different languages ['en', 'es], only the english version returns the file for the media entity. The media entity does in fact have a spanish translation however and should return the same file as it shows in /es/jsonapi but gatsby isn't resolving it and thus returning null.

Reproduction Link

Drupal dependency limits reproduction link.

Steps to Reproduce

On Drupal, set a Media type to translatable e.g. media_image
Create a media entity in english and then create a version of it in another language e.g. spanish
Review spanish jsonapi route: /es/jsonapi/media/image?include=field_media_image and notice the same file exists as the english version
In gatsby.config.js file in the gatsby-source-drupal settings add a filter "media--image": include=field_media_image
In the same section within the translatableEntities array, set "media--image".
Run gatsby develop and notice in the graphql tester that there are 2 media nodes for that type for each language but the spanish version is showing null for the localFile object.

Expected Result

localFile object should return the file for all language versions

Actual Result

localFile object is only being returned for the default language.

Environment

System:
    OS: macOS 10.15.7
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 16.3.0 - ~/.nvm/versions/node/v16.3.0/bin/node
    npm: 7.15.1 - ~/.nvm/versions/node/v16.3.0/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 91.0.4472.114
    Firefox: 90.0
    Safari: 14.0.3
  npmPackages:
    gatsby: ^3.8.1 => 3.9.1
    gatsby-plugin-gatsby-cloud: ^2.9.1 => 2.9.1
    gatsby-plugin-i18n: ^1.0.1 => 1.0.1
    gatsby-plugin-image: ^1.7.1 => 1.9.0
    gatsby-plugin-manifest: ^3.8.0 => 3.9.0
    gatsby-plugin-offline: ^4.8.0 => 4.9.0
    gatsby-plugin-react-helmet: ^4.8.0 => 4.9.0
    gatsby-plugin-sharp: ^3.9.0 => 3.9.0
    gatsby-plugin-styled-components: ^4.8.0 => 4.9.0
    gatsby-source-drupal: ^4.8.1 => 4.9.0
    gatsby-source-drupal-menu-links: ^2.0.0 => 2.0.0
    gatsby-source-filesystem: ^3.8.0 => 3.9.0
    gatsby-transformer-sharp: ^3.9.0 => 3.9.0
  npmGlobalPackages:
    gatsby-cli: 3.9.0

Config Flags

No response

apmsooner commented 3 years ago

When i console.log within exports.onCreateNode for each media entity... I notice the non default language ones are missing various relationships:

English version: { id: '0be6688a-2482-5667-b733-33e6a090c153', drupal_id: '47f0d257-4698-4976-a84e-2aa654817fe8', parent: null, drupal_parent_menu_item: undefined, children: [], drupal_internalmid: 7, drupal_internal__vid: 7, langcode: 'en', revision_created: '2021-07-06T15:31:14+00:00', revision_log_message: null, status: true, name: 'Westminster Dog Show', created: '2021-07-06T15:30:43+00:00', changed: '2021-07-19T03:37:50+00:00', default_langcode: true, revision_translation_affected: true, path: { alias: null, pid: null, langcode: 'en' }, content_translation_source: 'und', content_translationoutdated: false, relationships: { uidNODE: '5fcdae28-4387-5391-9a7b-3c115b0f56cc', thumbnail_NODE: '4ab2737b-bb3f-5429-ac49-c1f124740fbe', field_mediaimageNODE: '4ab2737b-bb3f-5429-ac49-c1f124740fbe', paragraph__hero_NODE: [ '0c9a6690-2c9f-518e-9f0f-862618dfdef7' ] }, internal: { type: 'mediabanner_image', contentDigest: '475a73e77cb11e7f22dbd093880da264', owner: 'gatsby-source-drupal', counter: 114 }, thumbnail: { alt: 'Westminster Dog Show', title: null, width: 1440, height: 450 }, field_media_image: { alt: 'Westminster Dog Show', title: '', width: 1440, height: 450 } }

Spanish version { id: '1b565514-3c97-5972-940a-20119c38604d', drupal_id: '47f0d257-4698-4976-a84e-2aa654817fe8', parent: null, drupal_parent_menu_item: undefined, children: [], drupal_internalmid: 7, drupal_internalvid: 7, langcode: 'es', revision_created: '2021-07-06T15:31:14+00:00', revision_log_message: null, status: true, name: 'El Westminster Dog Show', created: '2021-07-19T04:05:53+00:00', changed: '2021-07-19T04:21:55+00:00', default_langcode: false, revision_translation_affected: true, path: { alias: null, pid: null, langcode: 'es' }, content_translation_source: 'en', content_translation_outdated: false, relationships: { paragraphhero_NODE: [ '6ee1a3e6-48a3-5c4b-8b7a-bca28fd194ce' ] }, internal: { type: 'media__banner_image', contentDigest: '374c16f87725f9495855deef047814be', owner: 'gatsby-source-drupal', counter: 118 }, thumbnail: { alt: 'Westminster Dog Show', title: null, width: 1440, height: 450 }, field_media_image: { alt: 'Spanish Westminster Dog Show', title: '', width: 1440, height: 450 } }

Testing directly in jsonapi with the es prefix does show everything with relationships so not sure whats happening with gatsby-source-drupal.

{ "type": "media--banner_image", "id": "47f0d257-4698-4976-a84e-2aa654817fe8", "links": { "self": { "href": "http://local.h20:8888/es/jsonapi/media/banner_image/47f0d257-4698-4976-a84e-2aa654817fe8?resourceVersion=id%3A7" } }, "drupal_internalmid": 7, "drupal_internal__vid": 7, "langcode": "es", "revision_created": "2021-07-06T15:31:14+00:00", "revision_log_message": null, "status": true, "name": "El Westminster Dog Show", "created": "2021-07-19T04:05:53+00:00", "changed": "2021-07-19T04:21:55+00:00", "default_langcode": false, "revision_translation_affected": true, "path": { "alias": null, "pid": null, "langcode": "es" }, "content_translation_source": "en", "content_translation_outdated": false, "bundle": { "data": null, "links": { "self": { "href": "http://local.h20:8888/es/jsonapi/media/banner_image/47f0d257-4698-4976-a84e-2aa654817fe8/relationships/bundle?resourceVersion=id%3A7" } } }, "revision_user": { "data": null, "links": { "related": { "href": "http://local.h20:8888/es/jsonapi/media/banner_image/47f0d257-4698-4976-a84e-2aa654817fe8/revision_user?resourceVersion=id%3A7" }, "self": { "href": "http://local.h20:8888/es/jsonapi/media/banner_image/47f0d257-4698-4976-a84e-2aa654817fe8/relationships/revision_user?resourceVersion=id%3A7" } } }, "uid": { "type": "user--user", "id": "5e9fdc7d-f90e-4235-b5b1-38c99167408d" }, "thumbnail": { "type": "file--file", "id": "62291f52-70f4-4fd2-8490-7498582b1e54", "links": { "self": { "href": "http://local.h20:8888/jsonapi/file/file/62291f52-70f4-4fd2-8490-7498582b1e54" } }, "drupal_internalfid": 7, "langcode": "en", "filename": "110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg", "uri": { "value": "public://banners/2021-07/110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg", "url": "/sites/default/files/banners/2021-07/110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg" }, "filemime": "image/jpeg", "filesize": 80317, "status": true, "created": "2021-07-06T15:30:56+00:00", "changed": "2021-07-19T04:05:53+00:00", "meta": { "alt": "Westminster Dog Show", "title": null, "width": 1440, "height": 450 }, "uid": { "type": "user--user", "id": "5e9fdc7d-f90e-4235-b5b1-38c99167408d" } }, "field_media_image": { "type": "file--file", "id": "62291f52-70f4-4fd2-8490-7498582b1e54", "links": { "self": { "href": "http://local.h20:8888/jsonapi/file/file/62291f52-70f4-4fd2-8490-7498582b1e54" } }, "drupal_internal__fid": 7, "langcode": "en", "filename": "110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg", "uri": { "value": "public://banners/2021-07/110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg", "url": "/sites/default/files/banners/2021-07/110996_WKC-BIS-Image_Wasabi_1440x450_0.jpg" }, "filemime": "image/jpeg", "filesize": 80317, "status": true, "created": "2021-07-06T15:30:56+00:00", "changed": "2021-07-19T04:05:53+00:00", "meta": { "alt": "Spanish Westminster Dog Show", "title": "", "width": 1440, "height": 450 }, "uid": { "type": "user--user", "id": "5e9fdc7d-f90e-4235-b5b1-38c99167408d" } } },

apmsooner commented 3 years ago

This configuration also doesn't work when enabling media entity as translatable:

    disallowedLinkTypes: [
      `self`,
      `describedby`,
      `file--file`,
    ],
    filters: {
      "media--banner_image": `include=field_media_image`
    },
    languageConfig: {
      defaultLanguage: defaultLangKey,
      enabledLanguages: langs,
      translatableEntities: [
        `node--page`,
        `node--home_page`,
        `paragraph--link_list`,
        `paragraph--hero`,
        `paragraph--hero_carousel`,
        `menu_link_content--menu_link_content`,
        `media--banner_image`,
      ],
    },

I place file--file in disallowed link types to prevent downloading all files from drupal. Instead, i reference it with includes to get just the ones i need: e.g. include=field_media_image. This works great until I make the media entity translatable: e.g. media--banner_image. Even though everything is configured to be translatable on the drupal side, gatsby no longer pulls down ANY files with this configuration. Not sure how to resolve this with drupal files being non-translatable. I'm assuming the next language iteration is overwriting the english version with files as null leaving nothing.

apmsooner commented 3 years ago

@smthomas @KyleAMathews - I found the root of the issue i'm experiencing referencing the work with multilingual here: https://github.com/gatsbyjs/gatsby/pull/26720.

Problem** Drupal file--file is not translatable. When parent entity is translatable, its passing that language code to the function createNodeIdWithVersion and thus not resolving to a file object so the parent entity in english would have the file but the other languages are returning null. Depicted problem in this screenshot:

A possible solution is rewriting the langcode if type is file--file like so but not sure if this is the desired approach here or not?

const createNodeIdWithVersion = (id, type, langcode, revisionId, entityReferenceRevisions = []) => { var _getOptions$languageC;

// Files are not translatable in Drupal. if (type === 'file--file') { langcode = getOptions().languageConfig.defaultLanguage; }

// If the source plugin hasn't enabled translation then always just set langcode // to "undefined". let langcodeNormalized = getOptions().languageConfig ? langcode : und;

if (getOptions().languageConfig && !((_getOptions$languageC = getOptions().languageConfig) !== null && _getOptions$languageC !== void 0 && _getOptions$languageC.enabledLanguages.includes(langcodeNormalized))) { langcodeNormalized = getOptions().languageConfig.defaultLanguage; } // The relationship between an entity and another entity also depends on the revision ID if the field is of type // entity reference revision such as for paragraphs.

return isEntityReferenceRevision(type, entityReferenceRevisions) ? ${langcodeNormalized}.${id}.${revisionId || 0} : ${langcodeNormalized}.${id}; };

Desired result achieved with this modification:

apmsooner commented 3 years ago

As mentioned in the original PR for multilingual enhancement, there may be multiple entity types non translatable so wondering if it would make better sense to set in settings like:

nonTranslatableEntities: [ "file--file" ]

Then check type in that settings array to apply default language fallback.

Auspicus commented 3 years ago

@apmsooner I like the idea of including an option for nonTranslatableEntities. I actually just did that exact thing as a fork for this same issue.

apmsooner commented 3 years ago

@apmsooner I like the idea of including an option for nonTranslatableEntities. I actually just did that exact thing as a fork for this same issue.

@Auspicus - cool. Hey if your implementation is better than mine... all for whatever we can get in to the codebase. Feel free to take a look at the PR i posted: https://github.com/gatsbyjs/gatsby/pull/32548 and kindly review if this follows suit with what you did.

Auspicus commented 3 years ago

@apmsooner I just hacked it into the utils.js/handleReferences haha. Your setup is way better :D

apmsooner commented 3 years ago

@Auspicus - yeah i think i was heading that direction at first but then figured there were probably other entity types that might fall under this situation so might as well make it configurable to account for all. Thanks for the review!

gatsbyjs / gatsby