weird paths and database issues after installation from source code

[jpaulogg]

Running sioyek from the command line:

default_config_path: /usr/bin/prefs.config
Error: default_config_path: /usr/bin/prefs.config doesn't exist!
default_keys_path: /usr/bin/keys.config
Error: default_keys_path: /usr/bin/keys.config doesn't exist!
user_config_path: [ 0 ] /etc/xdg/sioyek/prefs_user.config
user_config_path: [ 1 ] /home/jpgg/.config/sioyek/prefs_user.config
user_config_path: [ 2 ] /home/jpgg/.config/.local/share/Sioyek/prefs_user.config
user_config_path: [ 3 ] /home/jpgg/.config/sioyek/prefs_user.config
user_keys_path: [ 0 ] /etc/xdg/sioyek/keys_user.config
user_keys_path: [ 1 ] /home/jpgg/.config/sioyek/keys_user.config
user_keys_path: [ 2 ] /home/jpgg/.config/.local/share/Sioyek/keys_user.config
user_keys_path: [ 3 ] /home/jpgg/.config/sioyek/keys_user.config
database_file_path: /home/jpgg/.config/.local/share/Sioyek/test.db
local_database_file_path: /home/jpgg/.config/.local/share/Sioyek/local.db
global_database_file_path: /home/jpgg/.local/share/Sioyek/shared.db
tutorial_path: /home/jpgg/.config/.local/share/Sioyek/tutorial.pdf
last_opened_file_address_path: /home/jpgg/.config/.local/share/Sioyek/last_document_path.txt
shader_path: /usr/bin/shaders
Error: shader_path: /usr/bin/shaders doesn't exist!
Creating shared memory block...
Shared memory created: this is the primary application.
Starting IPC server...
IPC server started.

Before installation, when running Sioyek from the build directory, I can open documents, but it doesn't seem to load data from shared.db even though I used the script provided to convert older databases to the new release format. I also tried using the --shared-database-path option, but it didn't solve the issue. After installing as root, sioyek fails to render any PDFs. The same happened when I installed the Linux binary from the alpha release.

Version sioyek-2.0.0_10 from Void Linux's package manager works fine (but I want to upgrade because I can't delete highlights with this version).

[ahrm]

This seems that you compiled sioyek and copied the executable to /usr/bin. This will not work because sioyek expects some files next to the executable. You should create a new build directory and copy the sioyek executable, shaders folder and keys.config and prefs.config files from the pdf_viewer dir to the build directory.

[jpaulogg]

I copied the executable to /usr/bin, the shaders folder to /usr/share/sioyek, and the configuration files to /etc/sioyek, as described in the sioyek(1) manual.

I also uncommented #define LINUX_STANDARD_PATHS in main.cpp, and now the paths seem correct. However, I am still experiencing issues with the database files.

[ahrm]

Can you share your old and new local.db and shared.db? Maybe there is an issue there.

[jpaulogg]

Here are the files. databases.zip

[ahrm]

These are just the old database files, where are the migrated database files?

[jpaulogg]

migrated_databases.zip

The script output:

$ database_migrator.py --old-shared-db backup/sioyek/shared.db  --new-shared-db /home/jpgg/Downloads/shared.db

Migrating table opened_books
Traceback (most recent call last):
  File "/home/jpgg/.local/bin/database_migrator.py", line 111, in <module>
    migrate(old_conn, new_conn)
  File "/home/jpgg/.local/bin/database_migrator.py", line 98, in migrate
    migrate_table(old_conn, new_conn, table)
  File "/home/jpgg/.local/bin/database_migrator.py", line 86, in migrate_table
    cursor.execute(f"INSERT OR REPLACE INTO {table_name} ({columns}) VALUES ({values})")
sqlite3.OperationalError: near ")": syntax error

$ database_migrator.py --old-shared-db backup/sioyek/local.db --new-shared-db /home/jpgg/Downloads/local.db

Migrating table document_hash
Traceback (most recent call last):
  File "/home/jpgg/.local/bin/database_migrator.py", line 111, in <module>
    migrate(old_conn, new_conn)
  File "/home/jpgg/.local/bin/database_migrator.py", line 98, in migrate
    migrate_table(old_conn, new_conn, table)
  File "/home/jpgg/.local/bin/database_migrator.py", line 86, in migrate_table
    cursor.execute(f"INSERT OR REPLACE INTO {table_name} ({columns}) VALUES ({values})")
sqlite3.OperationalError: near ")": syntax error

$ database_migrator.py --old-shared-db backup/sioyek/test.db --new-shared-db /home/jpgg/Downloads/test.db

Migrating table opened_books
Migrated 0 rows from opened_books
Migrating table marks
Migrated 0 rows from marks
Migrating table bookmarks
Migrated 0 rows from bookmarks
Migrating table highlights
Migrated 0 rows from highlights
Migrating table links
Migrated 0 rows from links
Migrating table document_hash
Migrated 0 rows from document_hash

[ahrm]

Have you followed the instructions here exactly? https://github.com/ahrm/sioyek/discussions/1156#discussioncomment-10614340 (specifically step 2, after deleting the old file you need to run the new version of sioyek once so that the new database files are created)

Note that the new shared.db file path should be one that is created by running the new sioyek version after moving the previous database files, not any arbitrary path.

[jpaulogg]

Have you followed the instructions here exactly?

Now I did it, but this didn't resolve the problem either. Here are the new migrated_databases.zip files and the command output:

$ cd ~/.local/share
$ python ~/.local/bin/database_migrator.py  --old-shared-db backup/sioyek/shared.db --new-shared-db sioyek/shared.db

Migrating table opened_books
Migrated 235 rows from opened_books
Migrating table marks
Migrated 126 rows from marks
Migrating table bookmarks
Migrated 5 rows from bookmarks
Migrating table highlights
Traceback (most recent call last):
  File "/home/jpgg/.local/bin/database_migrator.py", line 111, in <module>
    migrate(old_conn, new_conn)
  File "/home/jpgg/.local/bin/database_migrator.py", line 98, in migrate
    migrate_table(old_conn, new_conn, table)
  File "/home/jpgg/.local/bin/database_migrator.py", line 86, in migrate_table
    cursor.execute(f"INSERT OR REPLACE INTO {table_name} ({columns}) VALUES ({values})")
sqlite3.OperationalError: near "ser": syntax error

[ahrm]

The shared.db schema is still wrong. Wait I was assuming that you are trying to build the development branch of sioyek. Are you building the main branch?

[jpaulogg]

Yes, I was building the main branch. I am now trying to build the development branch, but I am encountering dependency issues. The latest error message is:

make: *** [Makefile:1696: TouchSettings.o] Error 1.

[ahrm]

This error doesn't tell me anything. Note that the development branch requires qt 6.7+.

[jpaulogg]

I'm having trouble compiling, so I'll try again later. Thanks for your help, and sorry for the confusion!

[jpaulogg]

I was able to build sioyek from the development branch (I had to symlink /usr/lib/qt6/bin/qmake-qt6 to /usr/bin/qmake-qt6). After following the 3 steps, the output for step 3 was:

Migrating table opened_books  
Migrated 235 rows from opened_books  
Migrating table marks  
Migrated 126 rows from marks  
Migrating table bookmarks  
Migrated 5 rows from bookmarks  
Migrating table highlights  
Traceback (most recent call last):  
  File "/home/jpgg/.local/bin/database_migrator.py", line 109, in <module>  
    migrate(old_conn, new_conn)  
  File "/home/jpgg/.local/bin/database_migrator.py", line 96, in migrate  
    migrate_table(old_conn, new_conn, table)  
  File "/home/jpgg/.local/bin/database_migrator.py", line 84, in migrate_table  
    cursor.execute(f"INSERT OR REPLACE INTO {table_name} ({columns}) VALUES ({values})")  
sqlite3.OperationalError: near "ser": syntax error

So, I’m only running into issues with the highlights. Here are the new database migrated_databases.zip

[jpaulogg]

After step 3, I replaced the new shared.db with the older one, and now it's working. So I'll close the github issue. Thanks for the help!