SPARQL endpoint occasionally returning 404 not found

fgiasson commented 9 years ago

Hi,

I am experiencing a strange behavior with the SPARQL endpoint. I have a process that send a series of SPARQL queries until a (big) dataset is entirely processed. What the SPARQL query does is to get slices of a dataset until it processed all entities of the dataset. Each query is send to the SPARQL HTTP endpoint.

However, during the process (some tests that leads to about 6000 queries), some of the queries does return a 404 not found page like this:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>Error HTTP/1.1 404 File not found</title>
  </head>
  <body>
    <h3>Error HTTP/1.1 404 File not found</h3><pre>
The requested URL was not found    URI  = '/sparql'
  </pre></body></html>

Then the following queries will most than likely work properly (in some cases not, since this issue is intermittent).

I am trying all kind of things to see what may be the problem, but I have a hard time to figure out what it may be. What I am wondering if if this occurs depending on the load of the Virtuoso server.

Do any of you have a hint about what could cause the SPARQL endpoint to return a 404 error for a specific query, but not subsequently for other queries?

Nothing appears in the log. The server doesn't crash, and the status(); seems all right.

Thanks!

HughWilliams commented 9 years ago

@fgiasson: Is this occurring with v7 or v6 ? Have you checked to see if there are locks or deadlocks occurring intermittently and which table they are on with the query:

SELECT TOP 10 *  FROM SYS_L_STAT  ORDER BY deadlocks DESC

You can also enable the CallStackOnException and PLDebug INI file params to try and obtain more details on any additional errors returned to the client as detailed in Handling Conditions In Virtuoso/PL Procedures.

fgiasson commented 9 years ago

Hi @HughWilliams,

This is using v7. The same process that was using ODBC instead of the SPARQL endpoint was working properly.

Thanks for the debugging hints, this is exactly what I was looking for. I will look deeper into this.

About the deadlocks, normally, if they occurs during a SPARQL query, a deadlock error is returned by the endpoint no (this is what I think remembering).

Will update this thread with more information once I have a better understanding of what is going on here.

Thanks!

fgiasson commented 9 years ago

@HughWilliams here are some updates. There are apparently no deadlocks when this occurs. However here are the locks for some of the GSPO indexes:

DB.DBA.RDF_QUAD   RDF_QUAD        29956
DB.DBA.RDF_QUAD   RDF_QUAD_GS     4968
DB.DBA.RDF_QUAD   RDF_QUAD_OP     1
DB.DBA.RDF_QUAD   RDF_QUAD_POGS   137943

Are these locks number normal? Continuing some testing on my side.

fgiasson commented 9 years ago

@HughWilliams,

I did enable CallStackOnException in virtuoso.ini. However, I am still only getting that HTML page when this error occurs:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>Error HTTP/1.1 404 File not found</title>
  </head>
  <body>
    <h3>Error HTTP/1.1 404 File not found</h3><pre>
The requested URL was not found    URI  = \'/sparql\'
  </pre></body></html>

Something I just noticed in virtuoso.log is 15:25:51 * Monitor: Locks are held for a long time

I am wondering if this may be the cause? In any case, I am sending hundred or thousands of subsequent queries to the /sparql/ endpoint, and eventually it sends a 404 error for a few subsequent queries, and eventually it resumes normally (process the query normally without sending the 404 error).

I am kind of stuck at the moment, not sure what I should test next.

fgiasson commented 9 years ago

@HughWilliams just so you know, I tested on another server with another dataset, and the same behavior happens (eventually it returns 404 after a few hundred of queries)

Also note that I am sending HTTP POST queries to the SPARQL endpoint

fgiasson commented 9 years ago

@HughWilliams some more information about that issue:

I enabled HTTP logging, then I saw that:
- when the system first get a 404 from the /sparql endpoint then all immediate subsequent queries returns a 404 as well. Eventually it restart (normally a second, or a few seconds after).
- the SPARQL endpoint can receive between 10 to 20 queries per second
- when the /sparql endpoint stop returning 404 errors, everything works as expected
- all queries are HTTP POST queries. But not sure if this is related or not.

The query that fails is:

      select ?s ?p ?o      
      where 
      {      
        {
          select distinct ?s
          from <http://some-dataset.com> 
          where 
          {
            ?s a ?type.
          } 

          limit 50
          offset 3000
        } 

        ?s ?p ?o
      }

Where the offset can be anything between 0 and 1 000 000. However, the issue normally start to happen when the offset is between 100 000 and 200 000, but sometimes it goes up to between 300 000 and 400 000.

HughWilliams commented 9 years ago

@fgiasson: We have tried to simulate this behaviour by replaying recordings from SPARQL endpoint activity but have been unable to do so.

Do all the queries use the same format as in the one above with just the limit & offset changing?

Are any of these datasets you are using public? As if so, and you provide the location of the dataset(s) to load, and provide a set of HTTP recording as detailed in HTTP Logging and Recording in Virtuoso, we can attempt to recreate in house. A copy of your virtuoso.ini file would also be useful to have, so the same can be used in our testing ...

fgiasson commented 9 years ago

@HughWilliams great, thanks for checking in this.

What I will do early next week is to create a EC2 AMI that I will share with you where the result that can reproduced, then I will tell you how to reproduce the issue on that instance.

Would that work?

HughWilliams commented 9 years ago

@fgiasson: That would work for me ...

fgiasson commented 9 years ago

@HughWilliams: I will wrap-up my work with OSF 3.3 (which will have an option to use unixODBC). Then I will test an installation that uses the SPARQL HTTP endpoints and reproduce the bug at that time. This means that you may not get the AMI until sometime next week.

andynolle commented 9 years ago

Hi all,

I've got the same behaviour and the error occurs on a series of ASK queries as well. I tried to find some log entries but can't find any.

Did you already found a solution for that?

Best Andy

andynolle commented 9 years ago

My guess is probably the parameter ServerThreads, because in the doc it is written: "If there are more concurrent requests, accepting the connections will be deferred until there is a thread ready to serve each."

Maybe it is also another parameter?

How can I find out the reason for the behaviour described above?

OpenDataAlex commented 8 years ago

I've hit the same issue, running 7.20.3215. My program will throw an error when the 404 hits. I've tested the query generated and it runs fine from the SPARQL Execution page without issue. The program I wrote is processing many SPARQL inserts into a specific graph (example query below). The 404 error hits randomly and causes the entire process to fail. Happy to provide logs/other details if someone can provide some guidance on what to post.

Sample queries:

INSERT
{
     GRAPH  <graph-address> { ?newSubject ? ?}
}
WHERE
{
     BIND(URI(<uri-string>) AS ?newSubject) .
}

AND

INSERT
{
     GRAPH  <graph-address> { ?newSubject ? ?newObject}
}
WHERE
{
     BIND(URI(<uri-string>) AS ?newSubject) .
     BIND(URI(<uri-string>) AS ?newObject) .
}

HughWilliams commented 8 years ago

@OpenDataAlex: So you are hitting the problem intermittently performing inserts via the /sparql endpoint, to which you have enabled SPARQL_UPDATE operations, as by default it is SPARQL_QUERY, i.e., read only?

OpenDataAlex commented 8 years ago

@HughWilliams : I've written an ETL process that turns database data into triples and then inserts them via SPARQL. I'm doing this because I have another process to handle updates by deleting records before reinserting them. I didn't know about the option to enable SPARQL_UPDATE - any documentation and/or can you tell me more? As the process stands now, the SPARQL queries are only inserts.

kidehen commented 8 years ago

A SPARQL Update is a combination of an INSERT and DELETE. There is no Update statement per se.

SPARQL_UPDATE is a ROLE in Virtuoso that's used to grant privileges to accounts in regards to INSERT and DELETE usage.

OpenDataAlex commented 8 years ago

@kidehen Thank you for the clarification. As I am testing functionality, the account I'm connecting to Virtuoso with has full privileges.

HughWilliams commented 8 years ago

@OpenDataAlex: Are your inserts being performed via the /sparql endpoint? As, if you have not set SPARQL_UPDATE, then you cannot, as it would be read only. Thus, how are they being performed, i.e., via SQL interface using JDBC/ODBC, etc., or other means?

Details on the SPARQL_UPDATE role can be found at Service Endpoint Security.

OpenDataAlex commented 8 years ago

@HughWilliams They are being done with the Jena JDBC driver thru Pentaho Data Integration directly on the SPARQL endpoint. Same for the deletes I'm performing.

HughWilliams commented 8 years ago

@OpenDataAlex: If you are connecting via the Virtuoso Jena Provider, which uses JDBC for connecting to the Virtuoso RDF Quad Store, then you are not connecting to the /sparql endpoint, in which case your issue is not related to this git issue as you are connecting via a different interface, and should thus open a new git issue for it ... Detail the condition in which a 404 is being reported, as I assume it must be coming from Pentaho, as the Jena Provider would not report such an error ...

OpenDataAlex commented 8 years ago

@HughWilliams Apologies! I'll create a ticket straight-away!

Edit: Ticket is: #565

nabito commented 5 years ago

It's 2019, and I also experiencing the same issue with @fgiasson I'm using open source virtuoso7.2.5.1, @HughWilliams has the issue been addressed?

The symptoms is exactly the same as being described above, intermittently returning HTTP 404 but in my case was performing SPARQL Insert query to SPARQL endpoint (/sparql).

The exact output is: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">\n<html>\n <head>\n <title>Error HTTP/1.1 404 File not found</title>\n </head>\n <body>\n <h3>Error HTTP/1.1 404 File not found</h3><pre>\nThe requested URL was not found URI = \'/sparql\'\n </pre></body></html>\n'

The number of insert queries is approx. ~600,000 and Interestingly, sometimes it all go through, but when it's not, ~100 requests were rejected with the above errors.

It's seems as if it's a kind of overflow problem on large consecutive insertion requests (depending also upon resource on the machine running Virtuoso).

If there is no real solution to this for now, I think I'll write the data into one large RDF file and do the bulk import instead.

amadanmath commented 5 years ago

Here's another report - I am getting the same error on Version 7.2.4.2.3217-pthreads as of Jul 25 2018.

I am not modifying the knowledgebase at all, just issuing a ton of simple SELECT queries one after another. For example, all of the statements issued today were of the form

PREFIX : <http://example.com/>
SELECT DISTINCT ?x WHERE { :foo :bar ?x. }

After a couple of hours, on a query no different from any others before it (except for :foo and :bar), I get the same 404 quoted above.

TallTed commented 5 years ago

@amadanmath - Updating to 7.2.5.1 or the latest 7.2.6-dev is strongly recommended, due to the significant number of changes since the 7.2.4.2 you're running.

Key questions toward resolving the issue include --

What tool is being used to issue the SELECT queries -- curl, a web browser, something else?
Is that tool running on the same host as the Virtuoso instance?
What OS (including version) is hosting the Virtuoso instance? If different, what OS hosts the querying tool?
Has any kernel tuning been done on the Virtuoso host?
How many triples/quads are in the instance?
Can we see the virtuoso.ini file?

amadanmath commented 5 years ago

I'll try to upgrade as soon as I am able to. (Will my databases be okay, or will I have to reimport them? They're rather huge...)

Python 3, requests 2.19.1
running on the same instance
Centos 7; both the database and virtuoso executable are on gpfs file systems (different ones)
I don't know about kernel tuning, I am not an admin
full Freebase (so I guess 1.9 billion ish?)
virtuoso-fb.ini (without comments or specific paths):

[Database]
DatabaseFile                    = /DBPATH/virtuoso.db
ErrorLogFile                    = /DBPATH/virtuoso.log
LockFile                        = /DBPATH/virtuoso.lck
TransactionFile                 = /DBPATH/virtuoso.trx
xa_persistent_file              = /DBPATH/virtuoso.pxa
ErrorLogLevel                   = 7
FileExtend                      = 200
MaxCheckpointRemap              = 2000
Striping                        = 0
TempStorage                     = TempDatabase
[TempDatabase]
DatabaseFile                    = /DBPATH/virtuoso-temp.db
TransactionFile                 = /DBPATH/virtuoso-temp.trx
MaxCheckpointRemap              = 2000
Striping                        = 0
[Parameters]
ServerPort                      = 1112
LiteMode                        = 0
DisableUnixSocket               = 1
DisableTcpSocket                = 0
MaxClientConnections            = 10
CheckpointInterval              = 60
O_DIRECT                        = 0
CaseMode                        = 2
MaxStaticCursorRows             = 5000
CheckpointAuditTrail            = 0
AllowOSCalls                    = 0
SchedulerInterval               = 10
DirsAllowed                     = ., /VIRTUOSOPATH/share/virtuoso/vad
ThreadCleanupInterval           = 0
ThreadThreshold                 = 10
ResourcesCleanupInterval        = 0
FreeTextBatchSize               = 100000
SingleCPU                       = 0
VADInstallDir                   = /VIRTUOSOPATH/share/virtuoso/vad/
PrefixResultNames               = 0
RdfFreeTextRulesSize            = 100
IndexTreeMaps                   = 256
MaxMemPoolSize                  = 200000000
PrefixResultNames               = 0
MacSpotlight                    = 0
IndexTreeMaps                   = 64
MaxQueryMem                     = 2G            ; memory allocated to query processor
VectorSize                      = 1000          ; initial parallel query vector (array of query operations) size
MaxVectorSize                   = 1000000       ; query vector size threshold.
AdjustVectorSize                = 0
ThreadsPerQuery                 = 4
AsyncQueueMaxThreads            = 10
NumberOfBuffers          = 5450000
MaxDirtyBuffers          = 4000000
[HTTPServer]
ServerPort                      = 8891
ServerRoot                      = /VIRTUOSOPATH/var/lib/virtuoso/vsp
MaxClientConnections            = 10
DavRoot                         = DAV
EnabledDavVSP                   = 0
HTTPProxyEnabled                = 0
TempASPXDir                     = 0
DefaultMailServer               = localhost:25
ServerThreads                   = 10
MaxKeepAlives                   = 10
KeepAliveTimeout                = 10
MaxCachedProxyConnections       = 10
ProxyConnectionCacheTimeout     = 15
HTTPThreadSize                  = 280000
HttpPrintWarningsInOutput       = 0
Charset                         = UTF-8
MaintenancePage                 = atomic.html
EnabledGzipContent              = 1
[AutoRepair]
BadParentLinks                  = 0
[Client]
SQL_PREFETCH_ROWS               = 100
SQL_PREFETCH_BYTES              = 16000
SQL_QUERY_TIMEOUT               = 0
SQL_TXN_TIMEOUT                 = 0
[VDB]
ArrayOptimization               = 0
NumArrayParameters              = 10
VDBDisconnectTimeout            = 1000
KeepConnectionOnFixedThread     = 0
[Replication]
ServerName                      = db-GPX001
ServerEnable                    = 1
QueueMax                        = 50000
[Striping]
Segment1                        = 100M, db-seg1-1.db, db-seg1-2.db
Segment2                        = 100M, db-seg2-1.db
[Zero Config]
ServerName                      = virtuoso (GPX001)
[Mono]
[URIQA]
DynamicLocal                    = 0
DefaultHost                     = localhost:8891
[SPARQL]
ResultSetMaxRows                = 10000
MaxQueryCostEstimationTime      = 400   ; in seconds
MaxQueryExecutionTime           = 60    ; in seconds
DefaultQuery                    = select distinct ?Concept where {[] a ?Concept} LIMIT 100
DeferInferenceRulesInit         = 0  ; controls inference rules loading
[Plugins]
LoadPath                        = /VIRTUOSOPATH/lib/virtuoso/hosting
Load1                           = plain, wikiv
Load2                           = plain, mediawiki
Load3                           = plain, creolewiki
Load4                   = plain, im

TallTed commented 5 years ago

@amadanmath

I'll try to upgrade as soon as I am able to. (Will my databases be okay, or will I have to reimport them? They're rather huge...)

Your databases should be fine with this update. There is no manual action required to update data structures, etc., between 7.2.4.2 and any of pre-built 7.2.5.1, pre-built 7.2.6-dev, or your own build from the latest 7.2.6-dev source.

We would recommend that you update both the binary (virtuoso-t) and all installed VADs (latest pre-built VADs are available here for VOS 7.2)

Of course, as with any update/upgrade, a backup is recommended just in case of discovery of a new issue, so you can return to the current known-state.

CheckpointInterval = 60

There have been instances of 404 errors being returned during CHECKPOINT() operations. Presuming minimal data change, you may want to disable automatic CHECKPOINT() (by changing that value from 60 to 0) and instead set up a cronjob or similar to run a command like isql localhost:1112 dba dba exec="CHECKPOINT();" at a low- or no-query time (e.g., daily at 3am).

Please do confirm that Virtuoso has 64GB RAM available to it, to match the NumberOfBuffers and MaxDirtyBuffers settings. (We would generally recommend a minimum of 20GB RAM for this deployment as described, with 1.9 Billion Triples; more is always better.)

rfm-targa commented 4 years ago

Greetings. I've also encountered this issue while inserting data. It does seem to happen during checkpoint. I am inserting data through the SPARQL endpoint. I use the Docker container with Virtuoso 7.2 (virtuoso-opensource-7:7.2), Python 3.7, requests 2.23.0 and SPARQLWrapper 1.8.4. The process that inserts the data does not abort due to this error, but the data that it tries to insert during this period will not be inserted and any operations that depend on the retrieval of that data will fail or return incomplete results. I've added retries and wait periods to my Python code to deal with the checkpoint period and hope that it is enough for it to keep trying sending and executing the same query until the checkpoint interval passes. When we insert data it is generally a lot in a short period and I think it is important to keep checkpoint enabled (currently every 10 minutes).

TallTed commented 4 years ago

@rfm-targa -- Based on your description of your workflow (which sounds like you insert data periodically but not constantly), I wonder whether it would make more sense overall for you to set the CheckpointInterval = 0, and programmatically execute a checkpoint() through an ODBC connection from Python after your bulk triple insert(s)...?

rfm-targa commented 4 years ago

Thank you for the response @TallTed. That seems like a good option, we will try to implement it.

farshad68 commented 3 years ago

We use this version of Virtuoso.

openlink/virtuoso-opensource-7:7.2.6-r10-g5a27710-alpine

Without any configuration and run in this way

sudo docker run -v /home/xxx/distinct_final_filtered_dbpedia_03_2021/:/opt/virtuoso-opensource/database/ -p 8890:8890 openlink/virtuoso-opensource-7:7.2.6-r10-g5a27710-alpine

Our algorithm does several queries at the endpoint. They are only select queries, and the problem is that after a while (no regular happening), we receive this error as a result, and for the following queries, the exact response received

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>Error HTTP/1.1 404 File not found</title>
  </head>
  <body>
    <h3>Error HTTP/1.1 404 File not found</h3><pre>
The requested URL was not found    URI  = '/sparql'
  </pre></body></html>

I have no idea what would be the reason for this or how I could resolve this. Can someone guide me with this issue? about the queries we run, they are a range of select queries, some simple like this

SELECT DISTINCT ?p1 WHERE { <http://dbpedia.org/resource/East_Timor> ?p1 <http://dbpedia.org/resource/Mari_Alkatiri> .  FILTER(  ?p1 != <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> && ?p1 != <http://dbpedia.org/ontology/wikiPageExternalLink> && ?p1 != <http://dbpedia.org/ontology/wikiPageWikiLink> )
 FILTER(strstarts(str(?p1),"http://dbpedia.org/ontology/")) 
 }"

and some long like this

SELECT  (count(distinct *) AS ?sum)
WHERE
  { ?s  a                     <http://dbpedia.org/class/yago/WikicatDutchLobbyists> ;
        a                     <http://dbpedia.org/class/yago/WikicatMinistersOfState(Netherlands)> ;
        a                     <http://dbpedia.org/class/yago/WikicatPresidentsOfTheClubOfMadrid> ;
        a                     <http://dbpedia.org/class/yago/Leader109623038> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchTradeUnionists> ;
        a                     <http://www.w3.org/2002/07/owl#Thing> ;
        a                     <http://www.wikidata.org/entity/Q82955> ;
        a                     <http://dbpedia.org/class/yago/WikicatNyenrodeBusinessUniversityAlumni> ;
        a                     <http://dbpedia.org/class/yago/Holder110180178> ;
        a                     <http://dbpedia.org/class/yago/HeadOfState110164747> ;
        a                     <http://dbpedia.org/class/yago/Persuader110418841> ;
        a                     <http://dbpedia.org/class/yago/PhysicalEntity100001930> ;
        a                     <http://dbpedia.org/class/yago/Executive110069645> ;
        a                     <http://dbpedia.org/class/yago/Official110372373> ;
        a                     <http://dbpedia.org/ontology/OfficeHolder> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchTradeUnionLeaders> ;
        a                     <http://dbpedia.org/class/yago/WikicatPartyLeadersOfTheLabourParty(Netherlands)> ;
        a                     <http://dbpedia.org/class/yago/Director110014939> ;
        a                     <http://dbpedia.org/class/yago/WikicatLivingPeople> ;
        a                     <http://www.wikidata.org/entity/Q24229398> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchAgnostics> ;
        a                     <http://dbpedia.org/class/yago/President110467179> ;
        a                     <http://dbpedia.org/class/yago/SkilledWorker110605985> ;
        a                     <http://dbpedia.org/class/yago/Politician110450303> ;
        a                     <http://dbpedia.org/class/yago/Person100007846> ;
        a                     <http://dbpedia.org/class/yago/Representative110522035> ;
        a                     <http://xmlns.com/foaf/0.1/Person> ;
        a                     <http://dbpedia.org/class/yago/WikicatPresidentsOfTheEuropeanCouncil> ;
        a                     <http://dbpedia.org/class/yago/WikicatPrimeMinistersOfTheNetherlands> ;
        a                     <http://schema.org/Person> ;
        a                     <http://dbpedia.org/class/yago/Minister110320863> ;
        a                     <http://dbpedia.org/class/yago/Owner110389398> ;
        a                     <http://dbpedia.org/class/yago/Organism100004475> ;
        a                     <http://dbpedia.org/class/yago/Whole100003553> ;
        a                     <http://dbpedia.org/ontology/Agent> ;
        a                     <http://dbpedia.org/class/yago/WikicatPeopleFromBergambacht> ;
        a                     <http://dbpedia.org/class/yago/LivingThing100004258> ;
        a                     <http://dbpedia.org/class/yago/Administrator109770949> ;
        a                     <http://dbpedia.org/class/yago/Head110162991> ;
        a                     <http://dbpedia.org/class/yago/YagoLegalActor> ;
        a                     <http://www.wikidata.org/entity/Q5> ;
        a                     <http://dbpedia.org/class/yago/Negotiator110351874> ;
        a                     <http://dbpedia.org/class/yago/Intellectual109621545> ;
        a                     <http://dbpedia.org/class/yago/Communicator109610660> ;
        a                     <http://dbpedia.org/class/yago/TradeUnionist110720964> ;
        a                     <http://dbpedia.org/ontology/Politician> ;
        a                     <http://dbpedia.org/class/yago/WikicatDeputyPrimeMinistersOfTheNetherlands> ;
        a                     <http://dbpedia.org/class/yago/Worker109632518> ;
        a                     <http://dbpedia.org/class/yago/WikicatLabourParty(Netherlands)Politicians> ;
        a                     <http://dbpedia.org/class/yago/Object100002684> ;
        a                     <http://dbpedia.org/class/yago/Scholar110557854> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchCorporateDirectors> ;
        a                     <http://dbpedia.org/class/yago/Agnostic109779124> ;
        a                     <http://dbpedia.org/class/yago/WikicatMinistersOfFinanceOfTheNetherlands> ;
        a                     <http://dbpedia.org/class/yago/YagoLegalActorGeo> ;
        a                     <http://www.ontologydesignpatterns.org/ont/dul/DUL.owl#Agent> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchPoliticians> ;
        a                     <http://dbpedia.org/class/yago/CausalAgent100007347> ;
        a                     <http://dbpedia.org/class/yago/Alumnus109786338> ;
        a                     <http://dbpedia.org/class/yago/WikicatDutchPeople> ;
        a                     <http://dbpedia.org/ontology/Person> ;
        a                     <http://dbpedia.org/class/yago/Officeholder110371450> ;
        a                     <http://dbpedia.org/class/yago/FinanceMinister110089892> ;
        a                     <http://dbpedia.org/class/yago/Lobbyist110268629> ;
        a                     <http://www.ontologydesignpatterns.org/ont/dul/DUL.owl#NaturalPerson> ;
        a                     <http://www.wikidata.org/entity/Q215627> .
    ?o  a                     <http://www.w3.org/2001/XMLSchema#string> .
    ?s (<http://dbpedia.org/ontology/deathPlace>/^<http://dbpedia.org/ontology/residence>)/<http://dbpedia.org/ontology/billed> ?o
  }

boutros commented 3 years ago

As it stands, I belive your best option is to modify your client to retry the query (after waiting some milliseconds) if it receives a 404 response from the SPARQL endpoint.

TallTed commented 3 years ago

@farshad68 —

First thing, please note that your sample "long" query does not seem likely to be doing what you want, as the sequence of a <TYPE_URI> ; a <TYPE_URI> ; ... is an implicit AND — and is equivalent to a sequence of a <TYPE_URI> , <TYPE_URI> , ... — so it will only match on ?s entities which are typed as all of those <TYPE_URI>, the COUNT of which I'm pretty sure will be zero!

I would suggest changing that to something like —

WHERE
  { ?s a ?type 
    FILTER ( ?type 
      IN ( <http://dbpedia.org/class/yago/WikicatDutchLobbyists> ;
           <http://dbpedia.org/class/yago/WikicatMinistersOfState(Netherlands)> ;
           <http://dbpedia.org/class/yago/WikicatPresidentsOfTheClubOfMadrid> ;
...
         ) )
  }

Toward analyzing the odd 404 responses — is your endpoint accessible to us? That would let us run some basic data collection queries to confirm some details of the active configuration (which should be running from the default virtuoso.ini found in the Docker image; with Virtuoso, there's really no such thing as "no configuration").

It would also be helpful for our testing if you can provide more detail of how you are running your queries, including any scripting you're using, along with the specific version of the associated scripting runtimes, and as many sample queries as you can reveal.

farshad68 commented 3 years ago

@TallTed Thank you for your hint about the query, but the query is the exact query we need (in this case, it has no result, and for us, it is ok). About the 404 error, yes, the endpoint is available, and it is

https://synthg-fact.dice-research.org/sparql

For running queries (the method is here ), we used to use “apache.Jena” but because it faced some strange issues (frozen while doing queries after a while and did not consider the timeout), we changed it to “org.apache.http.client” and we send our queries with GET , also algorithm run parallel threads to run these queries. ** [update about the situation the error happened] After the query faced with a 404 error, for around 6 minutes, the server responded to every query with a 404 error. After around 6 minutes, for the first error (which head to 404 for the first time ) server responded with

Virtuoso 42000 Error SR483: Stack Overflow

after restarting the docker, the server returned "timeout exception" but again, after several times, it returned stack overflow. for running the query, I used a web interface and this is the query

SELECT  (count(distinct *) AS ?sum)
WHERE
  { ?s  a                     <http://dbpedia.org/class/yago/WikicatAttemptedAssassinationSurvivors> ;
        a                     <http://dbpedia.org/class/yago/WikicatLycĂŠeLouis-le-GrandAlumni> ;
        a                     <http://dbpedia.org/class/yago/WikicatKnightsOfTheElephant> ;
        a                     <http://dbpedia.org/class/yago/Professional110480253> ;
        a                     <http://dbpedia.org/class/yago/ForeignMinister110103794> ;
        a                     <http://dbpedia.org/class/yago/Leader109623038> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchPeople> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchPeopleOfWorldWarI> ;
        a                     <http://www.w3.org/2002/07/owl#Thing> ;
        a                     <http://dbpedia.org/class/yago/WikicatPeopleFromParis> ;
        a                     <http://www.wikidata.org/entity/Q82955> ;
        a                     <http://dbpedia.org/class/yago/Holder110180178> ;
        a                     <http://dbpedia.org/class/yago/HeadOfState110164747> ;
        a                     <http://dbpedia.org/class/yago/PhysicalEntity100001930> ;
        a                     <http://dbpedia.org/class/yago/Executive110069645> ;
        a                     <http://dbpedia.org/class/yago/Official110372373> ;
        a                     <http://umbel.org/umbel/rc/PresidentOfOrganization> ;
        a                     <http://dbpedia.org/ontology/OfficeHolder> ;
        a                     <http://dbpedia.org/class/yago/Legislator110253995> ;
        a                     <http://dbpedia.org/class/yago/Aristocrat109807754> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchPoliticians> ;
        a                     <http://www.wikidata.org/entity/Q24229398> ;
        a                     <http://dbpedia.org/class/yago/President110467179> ;
        a                     <http://dbpedia.org/class/yago/SkilledWorker110605985> ;
        a                     <http://dbpedia.org/class/yago/WikicatTransportMinistersOfFrance> ;
        a                     <http://dbpedia.org/class/yago/Lawyer110249950> ;
        a                     <http://dbpedia.org/class/yago/Politician110450303> ;
        a                     <http://dbpedia.org/class/yago/WikicatRepublican-SocialistPartyPoliticians> ;
        a                     <http://dbpedia.org/class/yago/Senator110578471> ;
        a                     <http://dbpedia.org/class/yago/Person100007846> ;
        a                     <http://dbpedia.org/class/yago/Representative110522035> ;
        a                     <http://dbpedia.org/class/yago/WikicatPoliticiansOfTheFrenchThirdRepublic> ;
        a                     <http://xmlns.com/foaf/0.1/Person> ;
        a                     <http://schema.org/Person> ;
        a                     <http://dbpedia.org/class/yago/Minister110320863> ;
        a                     <http://dbpedia.org/class/yago/Survivor110681194> ;
        a                     <http://dbpedia.org/class/yago/Owner110389398> ;
        a                     <http://umbel.org/umbel/rc/Politician> ;
        a                     <http://dbpedia.org/class/yago/Organism100004475> ;
        a                     <http://dbpedia.org/class/yago/Socialist110618848> ;
        a                     <http://dbpedia.org/class/yago/Whole100003553> ;
        a                     <http://dbpedia.org/class/yago/WikicatPrimeMinistersOfFrance> ;
        a                     <http://dbpedia.org/class/yago/Knight110238375> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchForeignMinisters> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchLawyers> ;
        a                     <http://dbpedia.org/ontology/Agent> ;
        a                     <http://dbpedia.org/class/yago/LivingThing100004258> ;
        a                     <http://dbpedia.org/class/yago/WikicatCommunicationMinisters> ;
        a                     <http://dbpedia.org/class/yago/Administrator109770949> ;
        a                     <http://dbpedia.org/class/yago/Head110162991> ;
        a                     <http://dbpedia.org/class/yago/YagoLegalActor> ;
        a                     <http://www.wikidata.org/entity/Q5> ;
        a                     <http://dbpedia.org/class/yago/Negotiator110351874> ;
        a                     <http://dbpedia.org/class/yago/Intellectual109621545> ;
        a                     <http://dbpedia.org/class/yago/Lawgiver110249270> ;
        a                     <http://dbpedia.org/class/yago/Politician110451263> ;
        a                     <http://dbpedia.org/class/yago/Communicator109610660> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchSenatorsOfTheThirdRepublic> ;
        a                     <http://dbpedia.org/class/yago/WikicatPresidentsOfFrance> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchSocialists> ;
        a                     <http://dbpedia.org/ontology/Politician> ;
        a                     <http://dbpedia.org/class/yago/Worker109632518> ;
        a                     <http://dbpedia.org/class/yago/Object100002684> ;
        a                     <http://dbpedia.org/class/yago/Scholar110557854> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchMinistersOfCommerceAndIndustry> ;
        a                     <http://dbpedia.org/class/yago/WikicatFrenchMinistersOfDefence> ;
        a                     <http://dbpedia.org/class/yago/YagoLegalActorGeo> ;
        a                     <http://www.ontologydesignpatterns.org/ont/dul/DUL.owl#Agent> ;
        a                     <http://dbpedia.org/class/yago/CausalAgent100007347> ;
        a                     <http://dbpedia.org/class/yago/WikicatPoliticiansFromParis> ;
        a                     <http://dbpedia.org/class/yago/Alumnus109786338> ;
        a                     <http://dbpedia.org/class/yago/Unfortunate109630641> ;
        a                     <http://dbpedia.org/ontology/Person> ;
        a                     <http://dbpedia.org/class/yago/MaleAristocrat110285135> ;
        a                     <http://dbpedia.org/class/yago/Officeholder110371450> ;
        a                     <http://dbpedia.org/class/yago/WikicatGovernmentMinistersOfFrance> ;
        a                     <http://dbpedia.org/class/yago/Adult109605289> ;
        a                     <http://www.ontologydesignpatterns.org/ont/dul/DUL.owl#NaturalPerson> ;
        a                     <http://www.wikidata.org/entity/Q215627> .
    ?o  a                     <http://www.w3.org/2001/XMLSchema#string> .
    ?s (<http://dbpedia.org/ontology/birthPlace>/^<http://dbpedia.org/ontology/location>)/<http://dbpedia.org/ontology/language> ?o
  }

openlink / virtuoso-opensource

SPARQL endpoint occasionally returning 404 not found #299