search

maxlath / wikibase-cli

read and edit a Wikibase instance from the command line
MIT License
223 stars 24 forks source link

wd data crash reading aliases of 46k items #108

Closed rwst closed 4 years ago

rwst commented 4 years ago

I have 46k item IDs in foo and give this command:

wd data --simplify --props aliases <foo >bar.json

What I get (version is 11.3.1):

...
Q22320799
Q22320786
Q54809672
Q14864645
Q22313914
Q21122479
Q827489
    at ids.forEach.id (/usr/local/lib/node_modules/wikibase-cli/lib/fetch_and_log_ids_data.js:20:32)
    at Array.forEach (<anonymous>)
    at module.exports.ids (/usr/local/lib/node_modules/wikibase-cli/lib/fetch_and_log_ids_data.js:19:7)
    at Stream.end (/usr/local/lib/node_modules/wikibase-cli/lib/read_ids_from_stdin.js:22:5)
    at _end (/usr/local/lib/node_modules/wikibase-cli/node_modules/through/index.js:65:9)
    at Stream.stream.end (/usr/local/lib/node_modules/wikibase-cli/node_modules/through/index.js:74:5)
    at Stream.onend (internal/streams/legacy.js:44:10)
    at emitNone (events.js:111:20)
    at Stream.emit (events.js:208:7)
    at drain (/usr/local/lib/node_modules/wikibase-cli/node_modules/through/index.js:34:23)
rwst commented 4 years ago

Ah sry, I completely missed the start of stderr output:

/usr/local/lib/node_modules/wikibase-cli/lib/fetch_and_log_ids_data.js:20
    if (!isEntityId(id)) throw new Error(`invalid entity id: ${id}`)
                         ^

Error: invalid entity id: Q14762267
...

I can see the problem where to log the list of items, maybe make the output of the list to stderr optional, so that by default errors are more visible?

rwst commented 4 years ago

I see this is not intended to work? But putting the ids on one line was too much as well. I can give you the exact error tomorrow, afk now

rwst commented 4 years ago

I found two ways to do this more efficiently on my side so I'm not dependent on this atm. When I happen to stumble over it again I'll file a ticket more to the point.