search

namecoin / namecoin-legacy

Legacy client. New version here: https://github.com/namecoin/namecoin-core Note the release branch! - Official website:
https://namecoin.org
MIT License
448 stars 177 forks source link

name_filter command fails for large queries #234

Closed AyrA closed 6 years ago

AyrA commented 9 years ago

I noticed that the name_filter command fails when there is a large number of results to return

namecoind name_filter "^d/" 150 This command succeeds

namecoind name_filter "^d/" This command fails

I am using namecoin 0.3.80 on WIndows 7 x64. There is about 5 GB of memory available.

The error message I get is always: error: no response from server

The server instance does not prints any errors.

domob1812 commented 9 years ago

It seems like the call times out because it takes too long to process. In any case, name_filter is not very efficient - particularly on the old client that you use. As a first step, try using the new Namecoin Core client. This may already solve the issue for you.

AyrA commented 9 years ago

0.3.80 is the newest version advertised in the download section of the website

EDIT: and the forums

domob1812 commented 9 years ago

That's true, since the Namecoin Core client is still considered "experimental". Nevertheless, it runs without problems since half a year (and is also used by a major miner since then). I think that the issue in question is that the old client first loads the full name database into memory and then processes the regexp for name_filter. Namecoin Core applies the filter as it reads the database and needs much less memory. That's why I think that Namecoin Core should be able to resolve your issue.

Since the plan is to replace the old client with Namecoin Core "relatively soon" (basically as soon as a new UI is finished), we will not investigate more develoment effort into it unless absoluately needed (including improving the performance of name_filter). But if you have one, a pull request is, of course, gladly accepted. :)

@JeremyRand: Maybe we should change the website to "tentatively" recommend Namecoin Core if no "manage names" UI is needed?

AyrA commented 9 years ago

Do you have a windows binary available?

JeremyRand commented 9 years ago

@domob1812 I'm planning on proposing an "Implementations" page for namecoin.org, which will list both NamecoinQ and Namecoin Core (along with libcoind, NamecoinArmory, and a few others) along with brief descriptions of them. Would this satisfy your request?

@AyrA I don't believe we've posted official binaries. This is one reason it's not advertised on the website yet. I'm away at a conference this week, but dealing with posting official binaries of Namecoin Core is something I intend to tackle in the next month or so.

As an aside, if your use case is just grabbing all the names for a specific namespace, name_filter is way overkill, since it uses a full RegExp parser which is really slow. I think @domob1812 and I were discussing adding an RPC call for a prefix match... this might be a better solution to your problem, and would be much faster than name_filter, even on Namecoin Core. See https://github.com/namecoin/namecore/issues/16

AyrA commented 9 years ago

@JeremyRand I would also be happy, if I can extract everything from the namecoin blockchain. I can then feed it into an SQL database and search it this way.

JeremyRand commented 9 years ago

@AyrA You might be interested in libcoind. It's a full Bitcoin/Namecoin client (also supports a few other altcoins) that stores all the blockchain data (including Namecoin name data) in a SQLite database. https://github.com/libcoin/libcoin/ I believe it's used by Kraken for their audits.

JeremyRand commented 6 years ago

Doing some spring cleaning here. Namecoin Core Windows binaries have been officially available for quite a while. @AyrA can you please try with Namecoin Core, and open an issue in the Namecoin Core repo if it's still a problem? Closing this issue since Namecoin Legacy is not maintained. Thanks.