`theme.json` not processed: `Theme stylesheet detected. PHP Warning: Invalid argument supplied for foreach()`

strarsis commented 2 years ago

Bug Report

[x] Yes, I reviewed the contribution guidelines.
[x] Yes, more specifically, I reviewed the guidelines on how to write clear bug reports.

Describe the current, buggy behavior The (valid) theme.json is not processed by wp i18n, no translation strings are extracted from it. A PHP warning occurs:

Theme stylesheet detected.
PHP Warning:  Invalid argument supplied for foreach() in phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/ThemeJsonExtractor.php on line 209
PHP Stack trace:
PHP   1. {main}() /usr/local/bin/wp:0
PHP   2. include() /usr/local/bin/wp:4
PHP   3. include() phar:///usr/local/bin/wp/php/boot-phar.php:11
PHP   4. WP_CLI\bootstrap() phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php:27
PHP   5. WP_CLI\Bootstrap\LaunchRunner->process($state = class WP_CLI\Bootstrap\BootstrapState { private $state = ['context_manager' => class WP_CLI\ContextManager { ... }] }) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/bootstrap.php:78
PHP   6. WP_CLI\Runner->start() phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Bootstrap/LaunchRunner.php:28
PHP   7. WP_CLI\Runner->do_early_invoke($when = 'before_wp_load') phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php:1170
PHP   8. WP_CLI\Runner->run_command_and_exit($help_exit_warning = *uninitialized*) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php:125
PHP   9. WP_CLI\Runner->run_command($args = [0 => 'i18n', 1 => 'make-pot', 2 => '.', 3 => './resources/lang/theme.pot'], $assoc_args = [], $options = *uninitialized*) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php:440
PHP  10. WP_CLI\Dispatcher\Subcommand->invoke($args = [0 => '.', 1 => './resources/lang/theme.pot'], $assoc_args = [], $extra_args = []) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php:417
PHP  11. call_user_func:{phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php:491}($function_name = class Closure { virtual $closure = "$this->WP_CLI\Dispatcher\{closure}", public $static = ['callable' => [...]]; public $parameter = ['$args' => '<required>', '$assoc_args' => '<required>'] }, ...$parameters = variadic([])) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php:491
PHP  12. WP_CLI\Dispatcher\CommandFactory::WP_CLI\Dispatcher\{closure:phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php:97-104}($args = [0 => '.', 1 => './resources/lang/theme.pot'], $assoc_args = []) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php:491
PHP  13. call_user_func:{phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php:100}($function_name = [0 => class WP_CLI\I18n\MakePotCommand { protected $source = '/home/build/src/example.com/site/web/app/themes/theme'; protected $destination = './resources/lang/theme.pot'; protected $merge = [...]; protected $exceptions = [...]; protected $subtract_and_merge = NULL; protected $include = [...]; protected $exclude = [...]; protected $slug = 'theme'; protected $main_file_data = [...]; protected $skip_js = FALSE; protected $skip_php = FALSE; protected $skip_blade = FALSE; protected $skip_block_json = FALSE; protected $skip_theme_json = FALSE; protected $skip_audit = FALSE; protected $location = TRUE; protected $headers = [...]; protected $domain = 'theme'; protected $package_name = NULL; protected $file_comment = NULL; protected $project_type = 'theme' }, 1 => '__invoke'], ...$parameters = variadic([])) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php:100
PHP  14. WP_CLI\I18n\MakePotCommand->__invoke($args = [0 => '.', 1 => './resources/lang/theme.pot'], $assoc_args = []) phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php:100
PHP  15. WP_CLI\I18n\MakePotCommand->extract_strings() /home/build/.wp-cli/packages/vendor/wp-cli/i18n-command/src/MakePotCommand.php:285
PHP  16. WP_CLI\I18n\ThemeJsonExtractor::fromDirectory($dir = '/home/build/src/example.com/site/web/app/themes/theme', $translations = class Gettext\Translations { protected $headers = ['Project-Id-Version' => 'Theme 1.0.0', 'Report-Msgid-Bugs-To' => 'https://wordpress.org/support/theme/theme', 'Last-Translator' => 'FULL NAME <EMAIL@ADDRESS>', 'Language-Team' => 'LANGUAGE <LL@li.org>', 'MIME-Version' => '1.0', 'Content-Type' => 'text/plain; charset=UTF-8', 'Content-Transfer-Encoding' => '8bit', 'POT-Creation-Date' => '2022-05-10T21:12:15+02:00', 'PO-Revision-Date' => 'YEAR-MO-DA HO:MI+ZONE', 'X-Generator' => 'WP-CLI 2.6.0', 'X-Domain' => 'theme']; protected $translationClass = 'Gettext\\Translation'; private ${ArrayObject}storage = ['Theme' => class Gettext\Translation { ... }, 'https://www.example.com' => class Gettext\Translation { ... }, 'Test theme.' => class Gettext\Translation { ... }, 'Author' => class Gettext\Translation { ... }, 'https://www.web-design-hosting.eu' => class Gettext\Translation { ... }] }, $options = ['restrictFileNames' => [0 => 'theme.json'], 'include' => [], 'exclude' => [0 => 'node_modules', 1 => '.git', 2 => '.svn', 3 => '.CVS', 4 => '.hg', 5 => 'vendor', 6 => 'Gruntfile.js', 7 => 'webpack.config.js', 8 => '*.min.js'], 'extensions' => [0 => 'json']]) /home/build/.wp-cli/packages/vendor/wp-cli/i18n-command/src/MakePotCommand.php:676
PHP  17. WP_CLI\I18n\ThemeJsonExtractor::fromFile($file_or_files = [0 => '/home/build/src/example.com/site/web/app/themes/theme/.budfiles/bud/modules.json', 1 => '/home/build/src/example.com/site/web/app/themes/theme/.budfiles/bud/profile.json', 2 => '/home/build/src/example.com/site/web/app/themes/theme/composer.json', 3 => '/home/build/src/example.com/site/web/app/themes/theme/jsconfig.json', 4 => '/home/build/src/example.com/site/web/app/themes/theme/package.json', 5 => '/home/build/src/example.com/site/web/app/themes/theme/public/app.3be68fa733011d85f572.hot-update.json', 6 => '/home/build/src/example.com/site/web/app/themes/theme/public/app.daf54ef2be6639ef729e.hot-update.json', 7 => '/home/build/src/example.com/site/web/app/themes/theme/public/editor.3be68fa733011d85f572.hot-update.json', 8 => '/home/build/src/example.com/site/web/app/themes/theme/public/editor.daf54ef2be6639ef729e.hot-update.json', 9 => '/home/build/src/example.com/site/web/app/themes/theme/public/entrypoints.json', 10 => '/home/build/src/example.com/site/web/app/themes/theme/public/hmr.json', 11 => '/home/build/src/example.com/site/web/app/themes/theme/public/manifest.json', 12 => '/home/build/src/example.com/site/web/app/themes/theme/theme.json'], $translations = class Gettext\Translations { protected $headers = ['Project-Id-Version' => 'Theme 1.0.0', 'Report-Msgid-Bugs-To' => 'https://wordpress.org/support/theme/theme', 'Last-Translator' => 'FULL NAME <EMAIL@ADDRESS>', 'Language-Team' => 'LANGUAGE <LL@li.org>', 'MIME-Version' => '1.0', 'Content-Type' => 'text/plain; charset=UTF-8', 'Content-Transfer-Encoding' => '8bit', 'POT-Creation-Date' => '2022-05-10T21:12:15+02:00', 'PO-Revision-Date' => 'YEAR-MO-DA HO:MI+ZONE', 'X-Generator' => 'WP-CLI 2.6.0', 'X-Domain' => 'theme']; protected $translationClass = 'Gettext\\Translation'; private ${ArrayObject}storage = ['Theme' => class Gettext\Translation { ... }, 'https://www.example.com' => class Gettext\Translation { ... }, 'Test theme.' => class Gettext\Translation { ... }, 'Author' => class Gettext\Translation { ... }, 'https://www.web-design-hosting.eu' => class Gettext\Translation { ... }] }, $options = ['restrictFileNames' => [0 => 'theme.json'], 'include' => [], 'exclude' => [0 => 'node_modules', 1 => '.git', 2 => '.svn', 3 => '.CVS', 4 => '.hg', 5 => 'vendor', 6 => 'Gruntfile.js', 7 => 'webpack.config.js', 8 => '*.min.js'], 'extensions' => [0 => 'json']]) phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/IterableCodeExtractor.php:97
PHP  18. WP_CLI\I18n\ThemeJsonExtractor::fromString($string = '\n{\n  "$schema": "https://schemas.wp.org/trunk/theme.json",\n  "version": 2,\n  "settings": {\n    "color": {\n      "palette": [\n        {\n          "slug": "primary",\n          "color": "#525ddc",\n          "name": "Primary"\n        }\n      ],\n      "custom": false,\n      "customGradient": false,\n      "defaultPalette": false,\n      "defaultGradients": false\n    },\n    "custom": {\n      "spacing": {},\n      "typography": {\n        "font-size": {},\n        "line-height": {}\n      }\n    }', $translations = class Gettext\Translations { protected $headers = ['Project-Id-Version' => 'Theme 1.0.0', 'Report-Msgid-Bugs-To' => 'https://wordpress.org/support/theme/theme', 'Last-Translator' => 'FULL NAME <EMAIL@ADDRESS>', 'Language-Team' => 'LANGUAGE <LL@li.org>', 'MIME-Version' => '1.0', 'Content-Type' => 'text/plain; charset=UTF-8', 'Content-Transfer-Encoding' => '8bit', 'POT-Creation-Date' => '2022-05-10T21:12:15+02:00', 'PO-Revision-Date' => 'YEAR-MO-DA HO:MI+ZONE', 'X-Generator' => 'WP-CLI 2.6.0', 'X-Domain' => 'theme']; protected $translationClass = 'Gettext\\Translation'; private ${ArrayObject}storage = ['Theme' => class Gettext\Translation { ... }, 'https://www.example.com' => class Gettext\Translation { ... }, 'Test theme.' => class Gettext\Translation { ... }, 'Author' => class Gettext\Translation { ... }, 'https://www.web-design-hosting.eu' => class Gettext\Translation { ... }] }, $options = ['restrictFileNames' => [0 => 'theme.json'], 'include' => [], 'exclude' => [0 => 'node_modules', 1 => '.git', 2 => '.svn', 3 => '.CVS', 4 => '.hg', 5 => 'vendor', 6 => 'Gruntfile.js', 7 => 'webpack.config.js', 8 => '*.min.js'], 'extensions' => [0 => 'json'], 'file' => 'theme.json']) phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/IterableCodeExtractor.php:68
PHP  19. WP_CLI\I18n\ThemeJsonExtractor::get_fields_to_translate() phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/ThemeJsonExtractor.php:43
PHP  20. WP_CLI\I18n\ThemeJsonExtractor::extract_paths_to_translate($i18n_partial = ['title' => 'Style variation name', 'settings' => ['typography' => [...], 'color' => [...], 'blocks' => [...]], 'customTemplates' => [0 => [...]], 'templateParts' => [0 => [...]]], $current_path = *uninitialized*) phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/ThemeJsonExtractor.php:166
PHP  21. WP_CLI\I18n\ThemeJsonExtractor::extract_paths_to_translate($i18n_partial = 'Style variation name', $current_path = [0 => 'title']) phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/ThemeJsonExtractor.php:222
Success: POT file successfully generated!

(Valid )theme.json that causes the issue:

{
  "$schema": "https://schemas.wp.org/trunk/theme.json",
  "version": 2,
  "settings": {
    "color": {
      "palette": [
        {
          "slug": "primary",
          "color": "#525ddc",
          "name": "Primary"
        }
      ],
      "custom": false,
      "customGradient": false,
      "defaultPalette": false,
      "defaultGradients": false
    },
    "custom": {
      "spacing": {},
      "typography": {
        "font-size": {},
        "line-height": {}
      }
    },
    "spacing": {
      "padding": true,
      "units": ["px", "%", "em", "rem", "vw", "vh"]
    },
    "typography": {
      "customFontSize": false
    }
  }
}

Describe how other contributors can replicate this bug

Use a theme that contains this (valid )theme.json:

{
"$schema": "https://schemas.wp.org/trunk/theme.json",
"version": 2,
"settings": {
"color": {
  "palette": [
    {
      "slug": "primary",
      "color": "#525ddc",
      "name": "Primary"
    }
  ],
  "custom": false,
  "customGradient": false,
  "defaultPalette": false,
  "defaultGradients": false
},
"custom": {
  "spacing": {},
  "typography": {
    "font-size": {},
    "line-height": {}
  }
},
"spacing": {
  "padding": true,
  "units": ["px", "%", "em", "rem", "vw", "vh"]
},
"typography": {
  "customFontSize": false
}
}
}

Invoke wp i18n inside the theme directory: wp i18n make-pot . ./resources/lang/theme.pot Notice the PHP warning and that translation strings in theme.json are not extracted.

Describe what you would expect as the correct outcome No PHP warning and translation strings extracted from theme.json.

Let us know what environment you are running this on

OS:     Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64
Shell:  /bin/bash
PHP binary:     /usr/bin/php7.4
PHP version:    7.4.29
php.ini used:   /etc/php/7.4/cli/php.ini
MySQL binary:   /usr/bin/mysql
MySQL version:  mysql  Ver 8.0.29-0ubuntu0.20.04.3 for Linux on x86_64 ((Ubuntu))
SQL modes:
WP-CLI root dir:        phar://wp-cli.phar/vendor/wp-cli/wp-cli
WP-CLI vendor dir:      phar://wp-cli.phar/vendor
WP_CLI phar path:       /home/build/src/[...]/[...]/test-theme
WP-CLI packages dir:    /home/build/.wp-cli/packages/
WP-CLI global config:
WP-CLI project config:  /home/build/src/[...]/site/wp-cli.yml
WP-CLI version: 2.6.0

Provide a possible solution Probably some mismatch of expected theme.json data structure and extractor logic that needs to be fixed.

swissspidy commented 2 years ago

I just tested this with the current version in the main branch and also the improved version from #319 and could not reproduce this. String extraction works just fine.

So I suppose #306 fixed this.

Can you try installing the latest version of this package via wp package install git@github.com:wp-cli/i18n-command.git or by cloning this repository and see if it's still broken for you?

strarsis commented 2 years ago

@swissspidy: After installing the latest version of wp i18n (as you described), the PHP warning went away. However, the translations strings are not extracted from the theme.json:

{
  "$schema": "https://schemas.wp.org/trunk/theme.json",
  "version": 2,
  "settings": {
    "color": {
      "palette": [
        {
          "slug": "primary",
          "color": "#05536c",
          "name": "Dark turquoise"
        },
        {
          "slug": "turquoise-light",
          "color": "#1aaba8",
          "name": "Light turquoise"
        },
        {
          "slug": "sky-blue",
          "color": "#cff6ff",
          "name": "Sky blue"
        },
        {
          "slug": "beige",
          "color": "#eed9c8",
          "name": "Beige"
        },
        {
          "slug": "sand",
          "color": "#e2cb88",
          "name": "Sand"
        },
        {
          "slug": "grey-light",
          "color": "#fff8f9",
          "name": "Light grey"
        },
        {
          "slug": "purple",
          "color": "#733e90",
          "name": "Purple"
        },
        {
          "slug": "orange",
          "color": "#de6a16",
          "name": "Orange"
        },
        {
          "slug": "white",
          "color": "#fff",
          "name": "White"
        },
        {
          "slug": "black",
          "color": "#000",
          "name": "Black"
        }
      ],
      "custom": false,
      "customGradient": false,
      "defaultPalette": false,
      "defaultGradients": false
    },
    "custom": {
      "spacing": {},
      "typography": {
        "font-size": {},
        "line-height": {},
        "fontFamily": "El Messiri"
      }
    },
    "spacing": {
      "padding": true,
      "units": [
        "px",
        "%",
        "em",
        "rem",
        "vw",
        "vh"
      ]
    },
    "typography": {
      "customFontSize": false,
      "fontSizes": [
        {
          "slug": "regular",
          "size": "1.5rem",
          "name": "Regular"
        },
        {
          "slug": "huge",
          "size": "3rem",
          "name": "Huge"
        },
        {
          "slug": "larger",
          "size": "1.719rem",
          "name": "Larger"
        },
        {
          "slug": "large",
          "size": "1.375rem",
          "name": "Large"
        },
        {
          "slug": "medium",
          "size": "1.125rem",
          "name": "Medium"
        },
        {
          "slug": "smaller",
          "size": "0.813rem",
          "name": "Smaller"
        },
        {
          "slug": "small",
          "size": "0.938rem",
          "name": "Small"
        },
        {
          "slug": "tiny",
          "size": "0.75rem",
          "name": "Tiny"
        }
      ],
      "fontFamilies": [
        {
          "fontFamily": "'El Messiri', sans-serif",
          "slug": "el-messiri",
          "name": "El Messiri"
        }
      ]
    }
  },
  "styles": {
    "color": {
      "background": "var(--wp--preset--color--grey-light)",
      "text": "var(--wp--preset--color--primary)"
    }
  }
}

The palette and fontSizes names are not extracted.

swissspidy commented 2 years ago

This is fixed by #319 as far as I can tell.

swissspidy commented 2 years ago

Actually, on the main branch with this theme.json you provided I get the following POT file:

# Copyright (C) 2022 
# This file is distributed under the .
msgid ""
msgstr ""
"Project-Id-Version: foo theme\n"
"Report-Msgid-Bugs-To: https://wordpress.org/support/theme/foo-theme\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"POT-Creation-Date: 2022-05-11T06:06:44+00:00\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"X-Generator: WP-CLI 2.6.0\n"
"X-Domain: foo-theme\n"

#. Theme Name of the theme
msgid "foo theme"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Regular"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Huge"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Larger"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Large"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Medium"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Smaller"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Small"
msgstr ""

#: theme.json
msgctxt "Font size name"
msgid "Tiny"
msgstr ""

#: theme.json
msgctxt "Font family name"
msgid "El Messiri"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Dark turquoise"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Light turquoise"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Sky blue"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Beige"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Sand"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Light grey"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Purple"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "Orange"
msgstr ""

#: theme.json
msgctxt "Color name"
msgid "White"
msgstr ""

#: theme.json
#: styles/foo.json
msgctxt "Color name"
msgid "Black"
msgstr ""

Looks all correct to me.

strarsis commented 2 years ago

@swissspidy: Alright, the --include="app,resources" parameter made wp i18n ignore the theme.json. Adding theme.json to the include parameter made wp i18n also scan the theme.json again: --include="app,resources,theme.json"

That include excludes theme.json is probably intentional? Edit: This seems to be new, because when explicitly adding theme.json to include parameter, the foreach PHP warning appears again, but the translation strings are extracted, so it doesn't appear to cause issues in the output. Edit 2: That phar:///usr/local/bin/wp/vendor/wp-cli/i18n-command/src/ThemeJsonExtractor.php is shown in the path indicates that not the latest version of wp i18n from GitHub is used, but rather from latest release instead? I successfully ran the wp package install git@github.com:wp-cli/i18n-command.git command.

swissspidy commented 2 years ago

You can also try cloning this repository, running composer install and then vendor/bin/wp i18n make-pot to make sure you are really using the latest and greatest.

strarsis commented 2 years ago

@swissspidy: Yes! This approach worked. No PHP warnings anymore.

That theme.json has to be explicitly listed in --include is intentional? When --include is used, it acts as a whitelist, excluding everything else that is not listed?

swissspidy commented 2 years ago

I think so, yes

m1r0 commented 2 years ago

Hey! 👋

I've tried the above approach but it didn't work for me - I still get the error after installing the package manually (wp package install git@github.com:wp-cli/i18n-command.git).

What I've noticed is that I don't see the warning when using the binary from ./vendor/bin/wp. If I use the global wp command installed via Brew, the warning is there. 🦀

wojsmol commented 2 years ago

@m1r0 This is normal because when you are using a global wp command then you are using older version of i18n-command.

swissspidy commented 2 years ago

Not sure why the global wp isn't picking up that updated command for you. You might want to ask that question on the #wp-cli Slack.

Otherwise we're hopefully releasing a new proper version of this command (see https://github.com/wp-cli/i18n-command/milestone/23), maybe that helps.

The bug will be fixed there, so there's nothing else left to be done.

m1r0 commented 2 years ago

Got it, thanks!

strarsis commented 2 years ago

@swissspidy: A new release would be great! Currently I have to use the cloned vendor/bin/wp command.

Edit: That release would hopefully also include https://github.com/wp-cli/i18n-command/pull/316 for the new merge feature. that would be great.

strarsis commented 1 year ago

@swissspidy: Addendum: Besides theme.json, the patterns/ directory also has to be included explicitly when --include is used.

wp-cli / i18n-command

`theme.json` not processed: `Theme stylesheet detected. PHP Warning: Invalid argument supplied for foreach()` #322

Bug Report