MaxMind.Db.InvalidDatabaseException: Could not find a MaxMind Db metadata marker in this file (). Is this a valid MaxMind Db file?

[niemyjski]

I'm downloading and extracting (http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz) and loading the file from a stream. This works in .net 4.7. However when I run the same code on netcoreapp2.0 using the latest dependency I get the following exception.

[12/03/2017 00:30:37 > d55c6a: INFO] MaxMind.Db.InvalidDatabaseException: Could not find a MaxMind Db metadata marker in this file (). Is this a valid MaxMind Db file?
[12/03/2017 00:30:37 > d55c6a: INFO]    at MaxMind.Db.Reader.FindMetadataStart()
[12/03/2017 00:30:37 > d55c6a: INFO]    at MaxMind.Db.Reader..ctor(Buffer buffer)
[12/03/2017 00:30:37 > d55c6a: INFO]    at MaxMind.GeoIP2.DatabaseReader..ctor(Stream stream, IEnumerable`1 locales)
[12/03/2017 00:30:37 > d55c6a: INFO]    at MaxMind.GeoIP2.DatabaseReader..ctor(Stream stream)
[12/03/2017 00:30:37 > d55c6a: INFO]    at Exceptionless.Core.Geo.MaxMindGeoIpService.<GetDatabaseAsync>d__8.MoveNext() in 

[oschwald]

What is the md5sum for the database file that this is happening with? Could you provide a self-contained test case?

[niemyjski]

Sorry for the late reply. I don't know the hash of it. We have a job (https://github.com/exceptionless/Exceptionless/blob/master/src/Exceptionless.Core/Jobs/DownloadGeoIPDatabaseJob.cs) that downloads it to blob storage and I tried doing a rolling update to asp.net core in staging using both the previously downloaded file (from the legacy asp.net site) as well as a newly downloaded file and both errored. So I don't think it's something to do with the file itself. It has to be either how it's streamed from blob storage or the loading code while running on .net core.

[niemyjski]

The following tests pass: https://github.com/exceptionless/Exceptionless/blob/2cc38ba91a1efed38bc676fea2e00d5d3e58b9f2/tests/Exceptionless.Api.Tests/Plugins/GeoTests.cs on appveyor but I haven't tried them on linux or using blob storage as the backing source

[oschwald]

@eniqen, that exception will be thrown if the database is corrupt in some way. Are you sure it is the same issue that was reported here? What is the MD5 of the file you are trying to load?

[omar84]

good news, had this issue, solved it with adding the following line after reading the database file as a stream from Azure Blob Storage:

stream.Seek(0, SeekOrigin.Begin);

basically, if you're reading a stream from a blob, your function should look something like this:

CloudStorageAccount storageAccount = CloudStorageAccount.Parse("your_blob_storage_connection_string");
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlob = cloudBlobClient.GetBlobReferenceFromServer(new Uri("your_maxmind_db_file_blob_url"));
var stream = new MemoryStream();
cloudBlob.DownloadToStream(stream);
stream.Seek(0, SeekOrigin.Begin);
Reader reader = new Reader(stream);

hope this helps.

[oschwald]

I spent some time trying to reproduce this. The only way I was able to reproduce the issue is by changing the position of the the stream.

However, I don't think it is appropriate for the library to reset the position on the stream. Not all streams support seeking and there may be a legitimate reason why the database does not start at position 0 in the stream.

I would be happy to improve the documentation to clarify that the current position of the stream must be the start of the database, unless there are any better suggestions.

[oschwald]

I clarified the behavior in #51. Also, #44 made the behavior more consistent across seekable and non-seekable strings.