search

cldf / pycldf

python package to read and write CLDF datasets
https://cldf.clld.org
Apache License 2.0
15 stars 7 forks source link

Table type should be permitted to be None #84

Closed Anaphory closed 5 years ago

Anaphory commented 5 years ago

I created a new table in LexiRumah to collect Concept, Language tuples for which we don't have a form, but other information, following https://github.com/lessersunda/lexirumah-data/issues/81. This table (https://github.com/lessersunda/lexirumah-data/issues/81) does not have a dc:conformsTo entry, which is fine by cldf validate, but pycldf (which is part of the clics pipeline, so this matters right now) hiccups at

https://github.com/cldf/pycldf/blob/778333540ea757dccdff94b0c57dfb090e6db3b8/src/pycldf/dataset.py#L430

with

ValueError: Type None of table missing_forms.csv is not a valid term.
Anaphory commented 5 years ago

The error does not go higher in the call stack: The method is called from

File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/db.py", line 150, in <dictcomp>
  table_lookup = {t.url.string: t for t in ds.tables if ds.get_tabletype(t)}

which means that for the purposes of pyexibank, and thus the pipeline, it is perfectly valid to return the None.

Anaphory commented 5 years ago

There are however other complications: The code in the pull request is not sufficient.

lexibank load
Using configuration file at:
/home/gereon/.config/pylexibank/config.ini

INFO    processing lexirumah ...
Traceback (most recent call last):
  File "/home/gereon/.virtualenvs/clics/bin/lexibank", line 10, in <module>
    sys.exit(main())
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/__main__.py", line 139, in main
    sys.exit(parser.main())
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/clldutils/clilib.py", line 110, in main
    catch_all=catch_all, parsed_args=args)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/clldutils/clilib.py", line 82, in main
    self.commands[args.command](args)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/clldutils/clilib.py", line 35, in __call__
    return self.func(args)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/commands/misc.py", line 125, in load
    with_dataset(args, _load, default_to_all=True)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/commands/util.py", line 28, in with_dataset
    func(get_dataset(args, dataset.id), **vars(args))
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/commands/util.py", line 37, in _load
    db.load(ds)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/db.py", line 360, in load
    dataset = ds.cldf.wl
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/clldutils/misc.py", line 203, in __get__
    result = instance.__dict__[self.__name__] = self.fget(instance)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/dataset.py", line 355, in cldf
    return cldf.Dataset(self)
  File "/home/gereon/.virtualenvs/clics/lib/python3.7/site-packages/pylexibank/cldf.py", line 31, in __init__
    self.wl = Wordlist.from_metadata(md)
  File "/home/gereon/devel/pycldf/src/pycldf/dataset.py", line 339, in from_metadata
    return mod.cls(tablegroup)
  File "/home/gereon/devel/pycldf/src/pycldf/dataset.py", line 85, in __init__
    self.auto_constraints()
  File "/home/gereon/devel/pycldf/src/pycldf/dataset.py", line 169, in auto_constraints
    self.auto_constraints(table)
  File "/home/gereon/devel/pycldf/src/pycldf/dataset.py", line 189, in auto_constraints
    self._auto_foreign_keys(table, component=component, table_type=table_type)
  File "/home/gereon/devel/pycldf/src/pycldf/dataset.py", line 192, in _auto_foreign_keys
    assert (component is None) == (table_type is None)
AssertionError

For the purposes of the paper, I'll give it an unused type (ValueTable) until this issue becomes fixed.