sqlite3.OperationalError: no such table: conceptscheme

[mielvds]

I tried importing a skosfile with the following script:

wget https://raw.githubusercontent.com/viaacode/datamodels/main/description/production-method.skos.ttl
import_file production-method.skos.ttl https://data.hetarchief.be/id/production-method

However, I get the following database error:

0.104 Connecting to raw.githubusercontent.com (185.199.108.133:443)                                                                                                                                                                            
0.322 saving to 'production-method.skos.ttl'                                                                                                                                                                                                   
0.322 production-method.sk 100% |********************************|  2233  0:00:00 ETA                                                                                                                                                          
0.322 'production-method.skos.ttl' saved
0.660 2024-07-09 07:45:19,286 INFO sqlalchemy.engine.Engine BEGIN (implicit)
0.661 2024-07-09 07:45:19,287 INFO sqlalchemy.engine.Engine INSERT INTO conceptscheme (uri) VALUES (?)
0.661 2024-07-09 07:45:19,287 INFO sqlalchemy.engine.Engine [generated in 0.00010s] ('https://data.hetarchief.be/id/production-method',)
0.661 2024-07-09 07:45:19,287 INFO sqlalchemy.engine.Engine ROLLBACK
0.661 Traceback (most recent call last):
0.661   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
0.661     self.dialect.do_execute(
0.661   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
0.661     cursor.execute(statement, parameters)
0.661 sqlite3.OperationalError: no such table: conceptscheme

[goessebr]

I was able to fix it in my scaffold by specifying a target database. The default database is atramhasis.sqlite. I named my database atramhasis_scaffold.sqlite and I got the same error as you did. Fixed import for a dabase named atramhasis_scaffold.sqlite: import_file production-method.skos.ttl https://data.hetarchief.be/id/production-method --to sqlite:///atramhasis_scaffold.sqlite

Perhaps we should make the database a required property and not use a default database. Alternatively, there should be a clearer message. Any preference?

/cc @koenedaele @Wim-De-Clercq, @mielvds

[koenedaele]

Seems like using a default database is more harmful than helpful here, so I'm fine with making it a required property. Open to other suggestions as well.

[mielvds]

I would suggest to have a better error message and/or a log message that clarifies what database it's using. If so, it is not really necessary to require this property (and I would do this either way).

[koenedaele]

So, when importing a skos file, print a message similar to "Importing file {file} to database {database}"

Also, perhaps check if the database actually exists. I wonder if the database exists but is empty (because the migrations haven't run yet) or it doesn't exist and gets created on the fly.

[goessebr]

There is one other issue when using import_file production-method.skos.ttl https://data.hetarchief.be/id/production-method --to sqlite:///atramhasis_scaffold.sqlite.

Due to this validator, it is not possible to address the endpoint /providers/. The second argument https://data.hetarchief.be/id/production-method is not a valid uri_pattern. It would be beneficial to validate the uri_pattern for a valid input.

A valid import would look like this:

wget https://raw.githubusercontent.com/viaacode/datamodels/main/description/production-method.skos.ttl
import_file production-method.skos.ttl https://data.hetarchief.be/id/production-method/%s --to sqlite:///atramhasis_scaffold.sqlite

Scope:

• when importing a skos file, print a message similar to "Importing file {file} to database {database}"
• check if database exists: if not, print an error message
• check if database is not empty, if empty, print an error message
• check if uri_pattern is valid, if not print an error message