Closed CaseyRo closed 4 years ago
Have you tried WP CLI debugging flags? e.g. --debug=postmark
? Are the posts already previously synced? Do they have an ID?
hi @pjeby thanks for the quick response;
Debug: Starting sync of some starting thing.md (0.931s)
that's it.
So, it's just a markdown file like this:
---
ID: something-here
---
With no other content? That should produce an error message "Error: Content, title, and excerpt are empty". What version of wp-cli are you using?
Content:
---
Draft: true
Title: something that should get us started
ID: 12345
---
## Content Goes Here
If no `Title` was given, the above heading is stripped from the post body and used instead.
But if a `Title` *was* given, the heading remains in place.
WP-cli version 2.4.0
So tried a different PHP version; when using PHP 7.4 I'm becoming some other issues;
PHP Parse error: syntax error, unexpected 'fn' (T_FN), expecting identifier (T_STRING) or '{' in /home/deb96292n2/.wp-cli/packages/vendor/dirtsimple/imposer/src/Task.php on line 5
Parse error: syntax error, unexpected 'fn' (T_FN), expecting identifier (T_STRING) or '{' in /home/deb96292n2/.wp-cli/packages/vendor/dirtsimple/imposer/src/Task.php on line 5
Ah, you need to upgrade your imposer and dirtsimple/fun-factory
packages to get rid of that syntax error. Might be related to your actual problem, as well. ;-)
I'm not having any problems syncing that file in my environment, so I'm not sure where to go from here, aside from making sure you have the latest version of all the dirtsimple/*
packages (imposer, fun-factory, etc.) in your wp-cli packages directory (or global composer directory, if applicable). The exact procedure for upgrading is "it depends" because wp-cli has a very hairy way of dealing with vendor directories. It's very easy to end up with multiple copies of something installed, with an older version blocking a newer version, because there's more than one path things get loaded from.
So I've taken another few steps towards what I hope would make sense. Tried a few different things;
All didn't work, still got the PHP7.4 issue.
I dove into the imposer
code, and at the lines of the error message it was still using dirtsimple\fn
instead of dirtsimple\fun
, but I'm sure I've missed a few things next to that then, just changing them did get rid of the error messages - but then I'm stuck again at starting sync
, no progress.
Is there some hidden branch/update I'm missing here?
Please note, I'm keeping my setup as clean as possible - so just using a single vendor directory, one .wp-cli folder and having everything there. Composer doesn't have any dependancies of this project as global.
Edit:
2020-07-26T10:44:57.460314Z 30 [Note] Aborted connection 30 to db: 'wp_somethingwtdib' user: 'wp_somethingwtdib' host: 'localhost' (Got an error reading communication packets)
This is what mySQL gives back on the attempt to sync the file (and aborting it after a few minutes)
So, mysql hanging is definitely not a postmark issue. It sounds like your wp-cli environment has trouble connecting to mysql? Have you tried performing any wp
commands that use the database (e.g. wp option list
)?
As for the imposer code, see https://github.com/dirtsimple/imposer/commit/f6f02997cd03f85bc628849c26dd02d4b4970dfb -- if your version doesn't have those changes, then you haven't successfully updated. You need to go to the .wp-cli/packages
directory and run composer show
to check the exact versions you have, if they're not installed globally. For the dev projects (imposer and postmark) you should be able to see the exact commit of each, which you can then check against the head commits on github. It seems very odd to me that you would have an imposer version from last September or earlier, rather than from last October (which is the head on github).
The thing that seems really weird about the db connection is that ISTM you should have gotten the hang well before the "starting sync" message, if it was just a simple database error. So that's just weird. Postmark and Imposer only use the wpdb API, and only for a very few things.
Another suggestion: Wordpress rejects UUIDs that aren't of the form urn:uuid:...
. Postmark and Imposer don't care about this; Imposer has code to work around it. But there were bugs in early versions of that workaround code, so if you have an old enough Imposer (and if you have one that predates the fn -> fun change, you just might), then you might be experiencing that bug in some way.
Try changing the ID
of your document to be a valid URL scheme (from Wordpress's point of view). You can generate one with wp postmark uuid
. If this fixes the issue then you definitely have an outdated imposer. (Though you should also check for that directly with composer show
in the .wp-cli/packages
directory.)
Hi Pjeby; thanks for being so responsive!
WP option list
and other commands work fine, I can manage the WP instance fine through WP CLI, even on the shared hosting environment.I still get the error message on Task.php (and a few others if I change line 5 to be fun
instead of fn
):
Parse error: syntax error, unexpected 'fn' (T_FN), expecting identifier (T_STRING) or '{' in /Users/keesromkes/.wp-cli/packages/vendor/dirtsimple/imposer/src/Task.php on line 5
This doesn't happen when using a lower version of PHP than 7.4 btw, since well, it doesn't have this FN thing.
wp postmark uuid $filename
and that also works, so some functionality is doing things ;-)Just to hand you my composer show;
{
"description": "Installed community packages used by WP-CLI",
"homepage": "https://wp-cli.org/package-index/",
"license": "MIT",
"minimum-stability": "dev",
"name": "wp-cli/wp-cli",
"version": "2.4.0",
"authors": [
{
"email": "noreply@wpcli.org",
"name": "WP-CLI"
}
],
"config": {
"secure-http": true
},
"repositories": {
"wp-cli": {
"type": "composer",
"url": "https://wp-cli.org/package-index/"
}
},
"require": {
"dirtsimple/clean-yaml": "^0.1",
"dirtsimple/fun-factory": "^0.3.0",
"dirtsimple/imposer": "dev-master#8daa698db928560c6a5e8f9e328771da66e21ff4",
"dirtsimple/postmark": "dev-master",
"dirtsimple/yaml2json": "^1.2.1",
"guzzlehttp/promises": "^1.3",
"league/commonmark": "^1.0",
"league/commonmark-ext-smartpunct": "^1.1",
"league/commonmark-ext-strikethrough": "^1.0",
"league/commonmark-ext-table": "^2.1",
"rarst/wpdatetime": "^0.3.0",
"symfony/yaml": "^3.4",
"twig/twig": "^2.4",
"webuni/commonmark-attributes-extension": "^1.0",
"wp-cli/entity-command": "^2",
"wp-cli/eval-command": "^2"
},
"require-dev": {},
"prefer-stable": true
}
I've added all the required based on the contents of your packages, to make sure it all is synced and I have some form of control on the manual part.
That's not the output of composer show
, that's the contents of a composer.json. Totally different thing, as it's showing what you're asking for, not necessarily what's actually installed. composer show
lists the actually-installed versions of things.
It looks like imposer still has some fn
code in use
statements, and there are some errors with count(null)
in newer PHPs, so I've pushed a new version of imposer for that. It may be related to your hang, since it was part of imposer's scheduler/retry loop. If so, then upgrading to this version (dirtsimple/imposer@60d3f1be25) should fix it.
well that fixed the error messages on php 7.4, but is still 'stuck' on starting sync.
Here's the actual output of composer show;
dirtsimple/clean-yaml v0.1.3 Dump diff-friendly, readable YAML
dirtsimple/fun-factory 0.3.0 Fluently and efficiently compose functions, methods, and lambda-expression strings
dirtsimple/imposer dev-master 60d3f1b Modular Configuration Management for Wordpress
dirtsimple/postmark dev-master 3d53ff8 Sync WP content from Markdown files
dirtsimple/yaml2json v1.2.1 Full-fidelity YAML/JSON command-line converters
guzzlehttp/promises v1.3.1 Guzzle promises library
league/commonmark 1.5.3 Highly-extensible PHP Markdown parser which fully supports the CommonMark spec and Github-Flavored Markdown (GFM)
league/commonmark-ext-smartpunct v1.2.0 Intelligently converts ASCII quotes, dashes, and ellipses in Markdown to their Unicode equivalents
league/commonmark-ext-strikethrough v1.1.0 Strikethrough support for the PHP League's CommonMark Markdown parser
league/commonmark-ext-table v2.1.0 Table extension for league/commonmark
rarst/wpdatetime 0.3.1 Extension of PHP’s DateTime and DateTimeZone classes for WordPress context.
symfony/polyfill-ctype v1.18.0 Symfony polyfill for ctype functions
symfony/polyfill-mbstring v1.18.0 Symfony polyfill for the Mbstring extension
symfony/yaml v3.4.43 Symfony Yaml Component
twig/twig v2.13.0 Twig, the flexible, fast, and secure template language for PHP
webuni/commonmark-attributes-extension 1.0.0 The attributes extension adds a syntax to define attributes on the various HTML elements in CommonMark PHP impleme...
wp-cli/entity-command v2.0.7 Manage WordPress comments, menus, options, posts, sites, terms, and users.
wp-cli/eval-command v2.0.7 Executes arbitrary PHP code or files.
Still no success unfortunately.
Tried wp-cli with the full debug output; these are the last 6 lines, not sure if they are relevant for you;
Debug (bootstrap): Loaded WordPress (0.678s)
Debug (hooks): Processing hook "after_wp_load" with 1 callbacks (0.679s)
Debug (hooks): On hook "after_wp_load": Closure in file /Users/keesromkes/.wp-cli/packages/vendor/dirtsimple/postmark/command.php at line 9 (0.679s)
Debug (hooks): Processing hook "before_run_command" with 1 callbacks (0.679s)
Debug (hooks): On hook "before_run_command": WP_CLI\Bootstrap\RegisterDeferredCommands->add_deferred_commands() (0.679s)
Debug (bootstrap): Running command: postmark sync (0.679s)
Debug (postmark): Starting sync of mkdwn/index.md (0.697s)
On a positive note, processor usage dropped from ~90% to 30% (but still hanging)
I really don't know where to go from here without access to your environment. If I were in your environment, I would basically just start adding more WP_CLI::debug()
statements to the code to triangulate where the hang occurs.
This is as far as my limited PHP knowledge goes to track down the issue, please bare with me ;-)
line 82 in Kind.php, things stop. That is this line:
$id = yield $handler($doc);
The $handler itself contains:
array(2) {
[0]=>
string(32) "dirtsimple\Postmark\PostImporter"
[1]=>
string(8) "sync_doc"
}
and the $doc contains:
object(dirtsimple\Postmark\Document)#2060 (12) {
["loaded":protected]=>
bool(true)
["_cache_key":protected]=>
string(81) "/Volumes/iMac - Data/webroot/test/mkdwn/index.md:ed54eda80b80d991ae74f7ad415909e6"
["db":protected]=>
object(dirtsimple\Postmark\Database)#2045 (5) {
["cache":protected]=>
array(0) {
}
["allowCreate":protected]=>
bool(true)
["post_types":protected]=>
array(10) {
["post"]=>
int(1)
["page"]=>
int(1)
["attachment"]=>
int(1)
["revision"]=>
int(1)
["nav_menu_item"]=>
int(1)
["custom_css"]=>
int(1)
["customize_changeset"]=>
int(1)
["oembed_cache"]=>
int(1)
["user_request"]=>
int(1)
["wp_block"]=>
int(1)
}
["docs":protected]=>
object(dirtsimple\Imposer\Bag)#2056 (1) {
["storage":"ArrayObject":private]=>
array(2) {
["/Volumes/iMac - Data/webroot/test/mkdwn/index.md"]=>
*RECURSION*
["mkdwn/index.md"]=>
*RECURSION*
}
}
["results":protected]=>
object(dirtsimple\imposer\Pool)#2057 (2) {
["factory":protected]=>
object(Closure)#2058 (2) {
["this"]=>
*RECURSION*
["parameter"]=>
array(2) {
["$filename"]=>
string(10) "<required>"
["$pool"]=>
string(10) "<required>"
}
}
["storage":"ArrayObject":private]=>
array(1) {
["/Volumes/iMac - Data/webroot/test/mkdwn/index.md"]=>
object(dirtsimple\imposer\WatchedPromise)#2071 (3) {
["promise":protected]=>
object(GuzzleHttp\Promise\Promise)#2072 (6) {
["state":"GuzzleHttp\Promise\Promise":private]=>
string(7) "pending"
["result":"GuzzleHttp\Promise\Promise":private]=>
NULL
["cancelFn":"GuzzleHttp\Promise\Promise":private]=>
NULL
["waitFn":"GuzzleHttp\Promise\Promise":private]=>
NULL
["waitList":"GuzzleHttp\Promise\Promise":private]=>
NULL
["handlers":"GuzzleHttp\Promise\Promise":private]=>
array(1) {
[0]=>
array(3) {
[0]=>
object(GuzzleHttp\Promise\Promise)#2074 (6) {
["state":"GuzzleHttp\Promise\Promise":private]=>
string(7) "pending"
["result":"GuzzleHttp\Promise\Promise":private]=>
NULL
["cancelFn":"GuzzleHttp\Promise\Promise":private]=>
array(2) {
[0]=>
*RECURSION*
[1]=>
string(6) "cancel"
}
["waitFn":"GuzzleHttp\Promise\Promise":private]=>
NULL
["waitList":"GuzzleHttp\Promise\Promise":private]=>
array(1) {
[0]=>
*RECURSION*
}
["handlers":"GuzzleHttp\Promise\Promise":private]=>
array(0) {
}
}
[1]=>
NULL
[2]=>
object(Closure)#2073 (3) {
["static"]=>
array(1) {
["handler"]=>
string(42) "dirtsimple\imposer\Promise::deferred_throw"
}
["this"]=>
*RECURSION*
["parameter"]=>
array(1) {
["$reason"]=>
string(10) "<required>"
}
}
}
}
}
["handler":protected]=>
string(42) "dirtsimple\imposer\Promise::deferred_throw"
["checked":protected]=>
bool(false)
}
}
}
}
["_kind":protected]=>
string(7) "wp-post"
["slug":protected]=>
string(5) "mkdwn"
["project":protected]=>
object(dirtsimple\Postmark\Project)#2061 (6) {
["prefix_len":protected]=>
int(0)
["base":protected]=>
string(1) "/"
["loader":protected]=>
object(Twig\Loader\ArrayLoader)#2062 (1) {
["templates":"Twig\Loader\ArrayLoader":private]=>
array(0) {
}
}
["pdir":protected]=>
string(11) "//.postmark"
["prototypes":protected]=>
array(0) {
}
["env"]=>
object(Twig\Environment)#2064 (19) {
["charset":"Twig\Environment":private]=>
string(5) "UTF-8"
["loader":"Twig\Environment":private]=>
object(Twig\Loader\ChainLoader)#2063 (2) {
["hasSourceCache":"Twig\Loader\ChainLoader":private]=>
array(0) {
}
["loaders":"Twig\Loader\ChainLoader":private]=>
array(1) {
[0]=>
object(Twig\Loader\ArrayLoader)#2062 (1) {
["templates":"Twig\Loader\ArrayLoader":private]=>
array(0) {
}
}
}
}
["debug":"Twig\Environment":private]=>
bool(false)
["autoReload":"Twig\Environment":private]=>
bool(false)
["cache":"Twig\Environment":private]=>
object(Twig\Cache\NullCache)#2065 (0) {
}
["lexer":"Twig\Environment":private]=>
NULL
["parser":"Twig\Environment":private]=>
NULL
["compiler":"Twig\Environment":private]=>
NULL
["baseTemplateClass":"Twig\Environment":private]=>
string(14) "\Twig\Template"
["globals":"Twig\Environment":private]=>
array(0) {
}
["resolvedGlobals":"Twig\Environment":private]=>
NULL
["loadedTemplates":"Twig\Environment":private]=>
NULL
["strictVariables":"Twig\Environment":private]=>
bool(false)
["templateClassPrefix":"Twig\Environment":private]=>
string(15) "__TwigTemplate_"
["originalCache":"Twig\Environment":private]=>
bool(false)
["extensionSet":"Twig\Environment":private]=>
object(Twig\ExtensionSet)#2066 (15) {
["extensions":"Twig\ExtensionSet":private]=>
array(3) {
["Twig\Extension\CoreExtension"]=>
object(Twig\Extension\CoreExtension)#2068 (4) {
["dateFormats":"Twig\Extension\CoreExtension":private]=>
array(2) {
[0]=>
string(10) "F j, Y H:i"
[1]=>
string(7) "%d days"
}
["numberFormat":"Twig\Extension\CoreExtension":private]=>
array(3) {
[0]=>
int(0)
[1]=>
string(1) "."
[2]=>
string(1) ","
}
["timezone":"Twig\Extension\CoreExtension":private]=>
NULL
["escapers":"Twig\Extension\CoreExtension":private]=>
array(0) {
}
}
["Twig\Extension\EscaperExtension"]=>
object(Twig\Extension\EscaperExtension)#2069 (4) {
["defaultStrategy":"Twig\Extension\EscaperExtension":private]=>
bool(false)
["escapers":"Twig\Extension\EscaperExtension":private]=>
array(0) {
}
["safeClasses"]=>
array(0) {
}
["safeLookup"]=>
array(0) {
}
}
["Twig\Extension\OptimizerExtension"]=>
object(Twig\Extension\OptimizerExtension)#2070 (1) {
["optimizers":"Twig\Extension\OptimizerExtension":private]=>
int(-1)
}
}
["initialized":"Twig\ExtensionSet":private]=>
bool(false)
["runtimeInitialized":"Twig\ExtensionSet":private]=>
bool(false)
["staging":"Twig\ExtensionSet":private]=>
object(Twig\Extension\StagingExtension)#2067 (5) {
["functions":"Twig\Extension\StagingExtension":private]=>
array(0) {
}
["filters":"Twig\Extension\StagingExtension":private]=>
array(0) {
}
["visitors":"Twig\Extension\StagingExtension":private]=>
array(0) {
}
["tokenParsers":"Twig\Extension\StagingExtension":private]=>
array(0) {
}
["tests":"Twig\Extension\StagingExtension":private]=>
array(0) {
}
}
["parsers":"Twig\ExtensionSet":private]=>
NULL
["visitors":"Twig\ExtensionSet":private]=>
NULL
["filters":"Twig\ExtensionSet":private]=>
NULL
["tests":"Twig\ExtensionSet":private]=>
NULL
["functions":"Twig\ExtensionSet":private]=>
NULL
["unaryOperators":"Twig\ExtensionSet":private]=>
NULL
["binaryOperators":"Twig\ExtensionSet":private]=>
NULL
["globals":"Twig\ExtensionSet":private]=>
NULL
["functionCallbacks":"Twig\ExtensionSet":private]=>
array(0) {
}
["filterCallbacks":"Twig\ExtensionSet":private]=>
array(0) {
}
["lastModified":"Twig\ExtensionSet":private]=>
int(0)
}
["runtimeLoaders":"Twig\Environment":private]=>
array(0) {
}
["runtimes":"Twig\Environment":private]=>
array(0) {
}
["optionsHash":"Twig\Environment":private]=>
string(138) "["Twig\\Extension\\CoreExtension","Twig\\Extension\\EscaperExtension","Twig\\Extension\\OptimizerExtension"]:7:4:2.13.0:0:\Twig\Template:0"
}
}
["realpath":protected]=>
string(48) "/Volumes/iMac - Data/webroot/test/mkdwn/index.md"
["relative":protected]=>
string(48) "/Volumes/iMac - Data/webroot/test/mkdwn/index.md"
["filename":protected]=>
string(14) "mkdwn/index.md"
["body"]=>
string(174) "## Content Goes Here
If no `Title` was given, the above heading is stripped from the post body and used instead.
But if a `Title` *was* given, the heading remains in place.
"
["meta":"dirtsimple\Postmark\MarkdownFile":private]=>
NULL
["storage":"ArrayObject":private]=>
array(3) {
["ID"]=>
string(45) "urn:uuid:ba000658-4310-4215-86b8-3fdd22ad549d"
["Title"]=>
string(36) "something that should get us started"
["Resource-Kind"]=>
string(7) "wp-post"
}
}
Next place to put in debugging would be sync()
in src/PostImporter.php
, which should be called (by way of sync_doc()
) by this line you've stopped on. That's where the bulk of the sync work takes place. My guess is that it will get all the way through to return $postinfo->ref()
(in the main if
block) without any issues, and the actual hang is taking place afterwards.
One way to check that would be to edit imposer's src/Scheduler.php
to make the run()
function print each task before calling $task
, e.g. via:
diff --git a/src/Scheduler.php b/src/Scheduler.php
index 0348b69..23e0791 100644
--- a/src/Scheduler.php
+++ b/src/Scheduler.php
@@ -73,6 +73,7 @@ class Scheduler {
$progress = 0;
foreach ($todo as $task) {
$this->current = $task;
+ WP_CLI::debug("Trying task $task", "imposer");
$progress += $task->run();
$this->current = null;
if ( $this->restart_requested ) {
@@ -82,6 +83,7 @@ class Scheduler {
}
if ( ! $progress ) {
# We stalled, maybe due to unresolved references
+ WP_CLI::debug("No progress made on any task; rejecting deferred resources", "imposer");
foreach ($todo as $res) {
if ( $res instanceof Resource && $res->hasSteps() ) {
# Try to break the deadlock by rejecting a reference
If you run with --debug
and this spits out a nonstop series of "Trying task X" (possibly interspersed with "No progress made"), then that'll give a clue as to what's going wrong.
Hi @pjeby! Well, thanks for all the info!
I've worked through this, and have some good and bad news.
Let's start with the good - it works! (at least, content now flows into wordpress without stuck in the sync)
Bad news, I had to do something code wise. As you suggested, I looked into the sync()
function but it didn't finish the function completely.
It halted at the line: (in parent_id() function, line 36 in postimporter.php
)
return $doc->sync();
Once I commented out this line, it worked - but of course I would lose the parent/child relation because of this.
So it seems it's stuck in an endless loop because the following line doesn't return true, in all cases it returns FALSE
if ( ! $doc = $this->doc->parent() ) return null; # root, no parent
my current document setup:
index.md
folder1/index.md
folder1/folder2/index.md
folder1/folder2/something else.md
All 4 cases return 'false' for the equation mentioned above.
Let me know if you need some more info, at least I can start filling my content now!
I don't understand what you mean by it halting at that line; does it halt or have an infinite loop?
It sounds like you're saying that every document claims to have a parent, and that this recurses infinitely. Can you look at postmark's src/Document
and print the $filename
being generated by the parent()
method?
Hm. I just thought of something. Does your project directory have a .postmark
, _postmark
, .git
, or other marker of the top level?
Because I'm thinking that the issue here is that the search for a parent file isn't stopping.
Try this patch against the unmodified master:
diff --git a/src/Project.php b/src/Project.php
index a3649a4..d3d68ed 100644
--- a/src/Project.php
+++ b/src/Project.php
@@ -114,7 +114,9 @@ class Project {
$dir = dirname($filename);
if ( static::basename($filename) == 'index.md' ) {
if ( static::is_project($dir) ) return null;
- $dir = dirname($dir);
+ $newdir = dirname($dir);
+ if ( $newdir === $dir ) return null;
+ $dir = $newdir;
}
$filename = $dir == '.' ? 'index.md' : "$dir/index.md";
} while ( ! file_exists($filename) || ! filesize($filename) );
I think basically I never tested the case where there's neither a postmark directory alongside (or a checkout above) the imported files.
@pjeby so, your fix didn't really do anything - I got a different error message concerning that there is no title, content or excerpt
- but.
git init
this did the trick.
Happy camper here now, thank you very - very much for the result.
If I can be of help in 'fixing' this in some way, helping out testing specific functionality or whatever, just give me a ping - happy to support!
The no title/content/excerpt should only be if one of your index.md files is empty. In my local tests, the change appears to work so I'm checking it in. Let me know if there are any further issues.
Hiya! Love the idea you're doing here, and also the most complete and simple way thus far to handle this.
WP CLI works ok on the shared hosting I'm at - but for some reason I get no output whenever I try to sync (or tree) a post / set of posts.
--force, --skip-create and --porcelain all give no (hanging) output.
Is there a way to get some more debugging information, or activity of the process?
Edit: There is activity on the server, CPU is ticking up to 99% all of the time :-/