Unable to import data

[ghost]

Hello,

I have installed a new instance of iQvoc and I am unable to import any of the data I was able to import to my previous instance. Even the sample hobbies.nt file. The process starts and takes forever without any error and without succeeding. How can I troubleshoot? How can I know the error to solve? Is there anything related to workers or etc.?

Please advise as this is very critical. Thank you! Regards.

[matthiastz]

Can confirm, tried different .nt files - none of them could be imported. Process never ends, for some files it is already running since 4+ days (according to the UI), while I already restarted the app several times.

Something I found in the log: mDatabase\n name: crypted_password\n value_before_type_cast: f6b3bb45b0f2c536cc0b9d12dbc2347f5e0a173695525c44c2a4574533864560a550b510e90810c7709eae777064aa0644b6a3ed03a3712132fe28d6beeee0dc\n - !ruby/object:ActiveModel::AttributmDatabase\n name: crypted_password\n value_before_type_cast: f6b3bb45b0f2c536cc0b9d12dbc2347f5e0a173695525c44c2a4574533864560a550b510e90810c7709eae777064aa0644b6a3ed03a3712132fe28d6beeee0dc\n - !ruby/object:ActiveModel::Attributct:ActiveModel::Attribute::FromDatabase\n name: output\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: success\n value_before_type_cast: false\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: created_at\n value_before_type_cast: 2020-06-26 11:37:40.000000000 Z\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: updated_at\n value_before_type_cast: 2020-06-26 11:37:40.000000000 Z\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: finished_at\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: import_file\n value_before_type_cast: 8f0c67698467403ba6b16e8cc38f43c1.nt\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: publish\n value_before_type_cast: true\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: default_namespace\n value_before_type_cast: <deleted my URI here>/\n new_record: false\n active_record_yaml_version: 2\nfilename: \"my/remote/path/iqvoc/public/uploads/import/8f0c67698467403ba6b16e8cc38f43c1.nt\"\nuser: !ruby/object:User\n concise_attributes:\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: persistence_token\n value_before_type_cast: 8c998715388a7d9d9d6212d037d27e8d47d740ca93e0cc7b97a2ee02ff5594e2ff74456834ce319ad6acaf8ba489838414087f02a608b1e57bc8277a30f3a86c\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: active\n value_before_type_cast: true\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: failed_login_count\n value_before_type_cast: 0\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: id\n value_before_type_cast: 1\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: forename\n value_before_type_cast: Admin\n - !ruby/object:ActiveModel::Attribute::FromDatabase\n name: surname\n value_before_type_cast: IstratoStarted GET "/en/imports.html" for my.IP.address at 2020-06-26 13:39:30 +0200 Cannot render console from my.IP.address! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255

Especially the last line looks interesting: Cannot render console from <my.IP.address>! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255 Could this be a network / access rights problem? I have the iQvoc instance running on a remote server, connected via VPN to that network and accessing the web ui from my local PC.

[mjansing]

Hi, sorry for the inconvienence. How did you import your data? Please note that uploading an nt-file via webui only uploads an n-triples file and creates an corresponding import job. This jobs needs to be execute using an job worker (e.g. by running bundle exec rake jobs:workoff.

❯ be rake -T jobs
...
rake jobs:clear                 # Clear the delayed_job queue
rake jobs:work                  # Start a delayed_job worker
rake jobs:workoff               # Start a delayed_job worker and exit when all available jobs are complete

If you don't want to use webui for initial data loading, you could also use CLI: rake iqvoc:import:url URL=data/hobbies.nt NAMESPACE='http://hobbies.com/' (see https://github.com/innoq/iqvoc/wiki/Importing)

[matthiastz]

Hello @mjansing, yeah I initiated the import via the webui. Thanks for the hint with those rake jobs (work*), that definitely helped.

Still, I have problems importing SKOS vocabularies, most of the time I work with Turtle (.ttl) or RDF/XML (e.g. https://skos.um.es/unesco6/downloads.php). I tried Skosify & rdf2rdf for the conversion to NT (n-triples), result files always look valid to me.

When importing the unesco6 vocabulary I get a lot of the following log lines:

  WARN -- : SkosImporter: Unknown namespaces. Skipping <http://skos.um.es/unesco6/221214> skos:notation "2212.14"
  WARN -- : SkosImporter: Invalid origin. Skipping <http://skos.um.es/unesco6/221214> skos:notation "2212.14"

I think iQvoc does this for every triple of the vocabulary, since 0 concepts or collections are visible in the iQvoc gui or even in the mysql db. What is the problem with the unesco6 (@prefix unesco6: http://skos.um.es/unesco6/ .) namespace?

Thanks!