Short occurences of "File_Blocks_Index: Data file size does not match block size"

boldtrn commented 3 years ago

I run an overpass instance version 0.7.56.6 with minutely updates. For about 5-10 requests of every 10.000 requests I get the following error:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" lang="en"/>
  <title>OSM3S Response</title>
</head>
<body>

<p>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</p>
<p><strong style="color:#FF0000">Error</strong>: runtime error: open64: 0 Success /db/db/way_tags_local.bin.idx File_Blocks_Index: Data file size does not match block size </p>

</body>
</html>

I haven't found any posts of people where this happens only infrequently. I attached a few timestamps where this happens:

[2020-10-26T11:13:39.102Z]
[2020-10-26T13:22:30.933Z]
[2020-10-26T14:50:49.114Z]
[2020-10-26T14:51:07.144Z]
[2020-10-26T14:51:49.121Z]
[2020-10-26T14:58:17.389Z]
[2020-10-26T14:59:51.055Z]
[2020-10-26T15:00:15.765Z]
[2020-10-26T15:20:02.454Z]
[2020-10-26T15:23:32.219Z]
[2020-10-26T15:24:04.680Z]
[2020-10-26T15:24:53.837Z]
[2020-10-26T15:37:14.316Z]
[2020-10-26T15:37:38.945Z]
[2020-10-26T15:53:25.289Z]
[2020-10-26T17:28:49.782Z]
[2020-10-26T17:57:28.451Z]
[2020-10-26T17:57:44.742Z]
[2020-10-26T17:57:55.961Z]
[2020-10-26T18:08:16.917Z]
[2020-10-26T18:14:49.440Z]

At first I thought this might be due to data updates that for a very short time-frame the db is updated. But for this theory, the timestamps don't really match, for example:

[2020-10-26T17:57:28.451Z]
[2020-10-26T17:57:44.742Z]
[2020-10-26T17:57:55.961Z]

The number of issues is extremely low, so it's not a problem, but they popup in the logging and I was wondering what the reason for this could be or if this is a known issue?

BTW I run the docker container from: https://github.com/wiktorn/Overpass-API

mmd-osm commented 3 years ago

Can you reproduce the issue by capturing the query in question and trying again later?

boldtrn commented 3 years ago

This is only a temporary issue that seems to away after a short time. So if I try to access the same query later, it works fine.

Something that is also weird is that the http status code is 200 for these requests. Where I would have expected it to be 500. But that's maybe related to the setup of the wiktorn/Overpass-API repository?

drolbr commented 3 years ago

Could be a bug. The description so far does no suffice to pinpoint something.

TaoLeopard commented 2 years ago

Urgent Help please! I run into this issue right now with API.0.7.56.9 DB.2021.10.13. Once it gets into this state, it stays in this state. I can repro it for you on demand.

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

OSM3S Response

The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.

Error: runtime error: open64: 0 Success /opt/osm/database/way_tags_local.bin.idx File_Blocks_Index: Data file size does not match block size

@drolbr

TaoLeopard commented 2 years ago

My query:

[timeout:15][bbox:47.489304, -121.770561,47.511974, -121.70179];(way["highway"="path"];);out geom;

How do I mitigate this issue? Any way to reindex? Or refresh the DB?

mmd-osm commented 2 years ago

@TaoLeopard : please create a new issue rather than hijacking an unrelated issue (which is about sporadic failures). Don't forget to include as much information as possible how this issue can be reproduced. Describe your set up, your scenario, what data you run, etc.

TaoLeopard commented 2 years ago

OK, new issue opened: https://github.com/drolbr/Overpass-API/issues/638

drolbr / Overpass-API

Short occurences of "File_Blocks_Index: Data file size does not match block size" #602