search

Norconex / collector-core

Collector-related code shared between different collector implementations
http://www.norconex.com/collectors/collector-core/
Apache License 2.0
7 stars 15 forks source link

[http-collector 3.0.0-RC1] [MongoDataStoreEngine] Collection name for storetypes is too long #32

Closed sylvainroussy closed 2 years ago

sylvainroussy commented 2 years ago

Hi! After overriding class CrawlDocInfoService to customize the collections names with UUID suffix (we have one database per tenant and many differents cralers for each database), the MongoDataStoreEngine throws the following Exception:

Command failed with error 73 (InvalidNamespace): 'fully qualified namespace 6d1e7e9a-853f-4c3c-9f4d-a9b93ee49454_95_4_95_1.com.norconex.collector.core.store.impl.mongodb.MongoDataStoreEngine--storetypes is too long (max is 120 bytes)' on server 10.10.30.13:27017.

Facing to this issue, I have to override MongoDataStoreEngine but here again my custom MongoDataStoreEngine needs to put on package com.norconex.collector.core.store.impl.mongodb because MongoDataStore has no public methods (idFilter() for example) and it isn't a good way for me (because my custom classes are a part of my project, not norconex project).

This is an info, because I can place workarounds, but I think it's not a good way to deal with future Norconex updates...

sylvainroussy commented 2 years ago

Moved to collector-http project