dki-lab / Freebase-Setup

The last data dump of Freebase with introductory explanation of its schema
Creative Commons Zero v1.0 Universal
76 stars 15 forks source link

Failed to start virtuoso server #16

Open AiRyunn opened 1 year ago

AiRyunn commented 1 year ago

Hi, I have followed the instructions in REAMDE, but still have problems when starting the virtuoso server.

I downloaded and unzipped virtuoso_db.zip, and confirmed the structure of my directory as follows:

.
├── LICENSE
├── README.md
├── fix_freebase_literal_format.py
├── numeric_properties.txt
├── virtuoso.py
└── virtuoso_db
    ├── virtuoso.db
    ├── virtuoso.ini
    ├── virtuoso.lck
    └── virtuoso.log

1 directory, 9 files

Then I ran the command: python3 virtuoso.py start 3001 -d virtuoso_db, the process exits after ~15 seconds, showing the following output:

RUNNING: cat /proc/meminfo | grep MemTotal | awk '{print $2}'
527485136 KB free, using 9890346.299999999 buffers, 4945173.149999999 dirty buffers
[Database]
DatabaseFile = virtuoso_db/virtuoso.db
ErrorLogFile = virtuoso_db/virtuoso.log
LockFile = virtuoso_db/virtuoso.lck
TransactionFile = virtuoso_db/virtuoso.trx
xa_persistent_file = virtuoso_db/virtuoso.pxa
ErrorLogLevel = 7
FileExtend = 200
MaxCheckpointRemap = 2000
Striping = 0
TempStorage = TempDatabase

[TempDatabase]
DatabaseFile = virtuoso_db/virtuoso-temp.db
TransactionFile = virtuoso_db/virtuoso-temp.trx
MaxCheckpointRemap = 2000
Striping = 0

[Parameters]
ServerPort = 13001
LiteMode = 0
DisableUnixSocket = 1
DisableTcpSocket = 0
ServerThreads = 100 ; increased from 20
CheckpointInterval = 60
O_DIRECT = 1 ; increased from 0
CaseMode = 2
MaxStaticCursorRows = 100000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval = 10
DirsAllowed = .
ThreadCleanupInterval = 0
ThreadThreshold = 10
ResourcesCleanupInterval = 0
FreeTextBatchSize = 100000
PrefixResultNames = 0
RdfFreeTextRulesSize = 100
IndexTreeMaps = 256
MaxMemPoolSize = 200000000
PrefixResultNames = 0
MacSpotlight = 0
IndexTreeMaps = 64
NumberOfBuffers = 9890346.299999999
MaxDirtyBuffers = 4945173.149999999

[SPARQL]
ResultSetMaxRows = 50000
MaxQueryCostEstimationTime = 600 ; in seconds (increased)
MaxQueryExecutionTime = 180; in seconds (increased)

[HTTPServer]
ServerPort = 3001
Charset = UTF-8
ServerThreads = 15 ; increased from unknown

virtuoso_db/virtuoso.ini
==== Starting Virtuoso server for virtuoso_db on port 3001...
RUNNING: /home/ubuntu/data/virtuoso-opensource/bin/virtuoso-t +configfile virtuoso_db/virtuoso.ini +wait
The VDBMS server process terminated prematurely
after opening the database.

Then I checked virtuoso.log, which gave the following message:

                Fri Jul 14 2023
22:46:14 OpenLink Virtuoso Universal Server
22:46:14 Version 07.20.3229-pthreads for Linux as of Aug 15 2018
22:46:14 uses parts of OpenSSL, PCRE, Html Tidy

Unfortunately, the server exits almost immediately and there isn't much informative log output.

Any help in troubleshooting this problem would be greatly appreciated.

System Information

Lanskttee commented 1 year ago

I met the same problem. When I check the log, it says:

18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

AiRyunn commented 1 year ago

Just found the issue. The NumberOfBuffers and MaxDirtyBuffers in virtuoso.ini were set too high by virtuoso.py. That's why the process was killed by the os without any output. Lowered the buffer sizes and the problem is resolved.

AiRyunn commented 1 year ago

I met the same problem. When I check the log, it says:

18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is 148679688192 bytes.

Lanskttee commented 1 year ago

I met the same problem. When I check the log, it says: 18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is 148679688192 bytes.

It says the size is 23501950976. I just download it from another link in issues (from OneDrive).

AiRyunn commented 1 year ago

I met the same problem. When I check the log, it says: 18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is 148679688192 bytes.

It says the size is 23501950976. I just download it from another link in issues (from OneDrive).

I think we downloaded it from the same link. In that case, you might want to verify the database integrity, since it's even smaller than the zip file.

Lanskttee commented 1 year ago

Just found the issue. The NumberOfBuffers and MaxDirtyBuffers in virtuoso.ini were set too high by virtuoso.py. That's why the process was killed by the os without any output. Lowered the buffer sizes and the problem is resolved.

hi, I've lowered the size, but the problem still exists. Now I lower the size like this:

NumberOfBuffers = 608.227294921875 MaxDirtyBuffers = 7.602841186523437

ysu1989 commented 1 year ago

@Lanskttee for your issue, which is different from @AiRyunn as they rightfully pointed out, is most likely due to a corrupted DB file. I'd suggest to download again and make sure the download is 100% complete.

Lanskttee commented 1 year ago

@Lanskttee for your issue, which is different from @AiRyunn as they rightfully pointed out, is most likely due to a corrupted DB file. I'd suggest to download again and make sure the download is 100%

hi! I've downloaded it again and it didn't report this: 18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

But it still exits almost immediately. And it even didn't have a log. There is only a .ini file.

AiRyunn commented 1 year ago

@Lanskttee What is the full output of virtuoso.py and the files in the virtuoso_db directory?

Lanskttee commented 1 year ago

@Lanskttee What is the full output of virtuoso.py and the files in the virtuoso_db directory?

there is only a virtuoso.ini and virtuoso.db in the directory.

the output is:

$ python virtuoso.py start 3001 -d virtuoso_db 'D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t' is not recognized as an internal or external command, operable program or batch file. RUNNING: cat /proc/meminfo | grep MemTotal | awk '{print $2}' 16608660 KB free, using 608.227294921875 buffers, 7.602841186523437 dirty buffers [Database] DatabaseFile = virtuoso_db/virtuoso.db ErrorLogFile = virtuoso_db/virtuoso.log LockFile = virtuoso_db/virtuoso.lck TransactionFile = virtuoso_db/virtuoso.trx xa_persistent_file = virtuoso_db/virtuoso.pxa ErrorLogLevel = 7 FileExtend = 200 MaxCheckpointRemap = 2000 Striping = 0 TempStorage = TempDatabase

[TempDatabase] DatabaseFile = virtuoso_db/virtuoso-temp.db TransactionFile = virtuoso_db/virtuoso-temp.trx MaxCheckpointRemap = 2000 Striping = 0

[Parameters] ServerPort = 13001 LiteMode = 0 DisableUnixSocket = 1 DisableTcpSocket = 0 ServerThreads = 100 ; increased from 20 CheckpointInterval = 60 O_DIRECT = 1 ; increased from 0 CaseMode = 2 MaxStaticCursorRows = 100000 CheckpointAuditTrail = 0 AllowOSCalls = 0 SchedulerInterval = 10 DirsAllowed = . ThreadCleanupInterval = 0 ThreadThreshold = 10 ResourcesCleanupInterval = 0 FreeTextBatchSize = 100000 PrefixResultNames = 0 RdfFreeTextRulesSize = 100 IndexTreeMaps = 256 MaxMemPoolSize = 200000000 PrefixResultNames = 0 MacSpotlight = 0 IndexTreeMaps = 64 NumberOfBuffers = 608.227294921875 MaxDirtyBuffers = 7.602841186523437

[SPARQL] ResultSetMaxRows = 50000 MaxQueryCostEstimationTime = 600 ; in seconds (increased) MaxQueryExecutionTime = 180; in seconds (increased)

[HTTPServer] ServerPort = 3001 Charset = UTF-8 ServerThreads = 15 ; increased from unknown

virtuoso_db/virtuoso.ini ==== Starting Virtuoso server for virtuoso_db on port 3001... RUNNING: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t +configfile virtuoso_db/virtuoso.ini +wait

AiRyunn commented 1 year ago

@Lanskttee The error suggests that the command isn't recognized. Please make sure virtuoso-t is located in D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/ and executable.

Lanskttee commented 1 year ago

@Lanskttee The error suggests that the command isn't recognized. Please make sure virtuoso-t is located in D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/ and executable.

hi, I checked the mode and it is: -rwxrwxrwx+ 1 lans None 15456464 Aug 16 2018 virtuoso-t

But it still reports: /bin/sh: line 1: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t: cannot execute binary file: Exec format error

ysu1989 commented 1 year ago

it looks like you are on Windows. probably need to change all paths to Windows format.

AiRyunn commented 1 year ago

@Lanskttee Also virtuoso-t should be a binary executable file, not a shell script. You should be able to run it with ./virtuoso-t. You could try ./virtuoso-t --help to see if there is any output.

Lanskttee commented 1 year ago

@Lanskttee Also virtuoso-t should be a binary executable file, not a shell script. You should be able to run it with ./virtuoso-t. You could try ./virtuoso-t --help to see if there is any output.

it report like this: $ ./virtuoso-t --help bash: ./virtuoso-t: cannot execute binary file: Exec format error

Lanskttee commented 1 year ago

it looks like you are on Windows. probably need to change all paths to Windows format.

hi! When I changed the slash, it just reports like this: /bin/sh: line 1: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource//bin//virtuoso-t: cannot execute binary file: Exec format error

And backslash like this: /bin/sh: line 1: D:pycharmKB-BINDER-mainfujianvirtuoso-opensource.x86_64-generic_glibc25-linux-gnuvirtuoso-opensourcebinvirtuoso-t: command not found It disappeared in the output.

BTW, I do on Window, but I use cygwin to run it.

AiRyunn commented 1 year ago

@Lanskttee I'm not familiar with Windows, but this seems to be a problem with virtuoso installation. You can try reinstalling virtuoso, or building virtuoso 7.2.5 from source (https://github.com/openlink/virtuoso-opensource/releases).

ISPZ commented 1 year ago

we meet the error

Error HTTP/1.1 404 File not found
The requested URL was not found    URI  = '/'

when open http://localhost:3001

slintday commented 1 year ago

我们遇到了错误

Error HTTP/1.1 404 File not found
The requested URL was not found    URI  = '/'

打开时 http://localhost:3001

Is this problem solved? I also encountered the same problem.