Question in regards lookup ranges

[angelenriquep]

Hello mr. Achilleas, hope allg ood, just a quick one: Is it the right approach to use UUIDs in range queries? or just for demo purposes? thanks!

[achilleasa]

Hi there!

For the use-case decribed in the book, we exploit the fact that randomly generated V4 UUIDs are unlikely to conflict (though the code tries to handle such edge cases) and that they tend to be uniformly distributed in the UUID space. The latter attribute allows us to effectively partition them into approximately equal-sized sets based on the number of workers we want to use.

Note that the UUIDs are used as a primary keys for each row in the database and therefore we can benefit from the PK index to efficiently answer range queries. In addition, cockroachDB has native support for UUID types (stored internally as a 128bit integers) which is also another reason why I opted to use them for performing range queries.

The caveat is that they do take a lot of space (especially on DB systems that store them as strings) so depending on your particular use-case you might be able to use a different primitive (e.g. a 64bit integer) as long as you can ensure a more or less uniform distribution of values in the available value range.

[angelenriquep]

Perfect thanks a lot Mr Achilleas!

[angelenriquep]

Hello Mr Achilleas, hope ou are doing well, mega quick one, where is the link provider service defined? I mean the one that in in charge to pull from the graph api and feed the input sink of the pipe:

Thanks a lot in advance!

[achilleasa]

Hi there!

The crawler design is outlined in chapter 7 and you can find its complete implementation in crawler.go.

If you take a closer look at its implementation you will notice that the signature for the Crawl method is:

Crawl(ctx context.Context, linkIt graph.LinkIterator) (int, error)

The caller is responsible for providing a graph.LinkIterator instance as the second argument. The graph package was designed in Chapter 6 and the interface definition can be found here. The interface looks like this:

type LinkIterator interface {
    Iterator

    // Link returns the currently fetched link object.
    Link() *Link
}

Chapter 11 describes the crawler service implementation that wires all components developed in the previous chapters together into a fully functioning service. Remember that the LinkGraphAPI provides a method for querying the graph and getting back a LinkIterator (implemented under the hood as a streaming gRPC call). All the service does is to fetch the iterator and pass it through to the Crawl method.

Hope this answers your question.

[angelenriquep]

Again mega thanks a lot Mr Achilleas, incredible book, best I read