Creating names.yaml with ORCID_2021_10_summaries.tar.gz fails

[chriz-uniba]

Package version (if known): invenio-app-rdm: v9.1.3

Describe the bug

When following the instructions here: https://inveniordm.docs.cern.ch/customize/vocabularies/names/#creating-a-namesyaml-file

we get for the convert command many lines with:

TarReader.read: Record not found in XML entry.
TarReader.read: Record not found in XML entry.
OrcidTransformer: Name not found in ORCiD entry.
TarReader.read: Record not found in XML entry.
TarReader.read: Record not found in XML entry.
TarReader.read: Record not found in XML entry.
TarReader.read: Record not found in XML entry.

end result:

Vocabulary names converted. Total items 12638934. 
12306834 items succeeded
332100 contained errors
0 were filtered.

we get for the import command

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/contrib/names/datastreams.py", line 104, in write
    current = self._resolve(vocab_id)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/contrib/names/datastreams.py", line 95, in _resolve
    return self._service.resolve(self._identity, id_=id_, id_type=self._scheme_id)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/contrib/names/services.py", line 45, in resolve
    raise PIDDoesNotExistError(pid_type=id_type, pid_value=id_)
invenio_pidstore.errors.PIDDoesNotExistError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/bin/invenio", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/cli.py", line 127, in import_vocab
    success, errored, filtered = _process_vocab(config, num_samples)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/cli.py", line 81, in _process_vocab
    for result in ds.process():
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/datastreams/datastreams.py", line 61, in process
    yield self.write(transformed_entry)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/datastreams/datastreams.py", line 104, in write
    writer.write(stream_entry)
  File "/usr/local/lib/python3.9/site-packages/invenio_vocabularies/contrib/names/datastreams.py", line 112, in write
    return StreamEntry(self._service.create(self._identity, entry))
  File "/usr/local/lib/python3.9/site-packages/invenio_records_resources/services/uow.py", line 324, in inner
    res = f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 293, in create
    return self._create(self.record_cls, identity, data, uow=uow, expand=expand)
  File "/usr/local/lib/python3.9/site-packages/invenio_records_resources/services/uow.py", line 328, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/invenio_records_resources/services/records/service.py", line 308, in _create
    data, errors = self.schema.load(
  File "/usr/local/lib/python3.9/site-packages/invenio_records_resources/services/records/schema.py", line 83, in load
    valid_data = self.schema(context=context, **schema_args).load(data)
  File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 722, in load
    return self._do_load(
  File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 909, in _do_load
    raise exc
marshmallow.exceptions.ValidationError: {'family_name': ['Field may not be null.']}

Expected behavior

• for the convert command, in discord we got the information, that the errors might should got filtered out (due to missing family_names; see https://discord.com/channels/692989811736182844/1013713642857967626/1014184903044239382)

• for the import I would expect that it is not crashing and generating the expected names-vocabulary.

Additional context

• We found a lot of family_name null values when taking a look at the created/converted names.yaml
• As always, it is a pity that it seems to be the fact, that people without a family_name have to filtered out and cannot be included in the names.yaml: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/

[ppanero]

note for future: shuold give more info about entries