crawler-commons url-frontier issues

crawler-commons / url-frontier

API definition, resources and reference implementation of URL Frontiers

Apache License 2.0

46 stars 12 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

URLFrontier AckMessage does not contain IDs of ACKed URLInfo

#62 FelixEngl closed 2 years ago
3
Update version of gRPC to 1.50.0

#61 jnioche closed 2 years ago
0
RocksDB backend - faster restarts, implements #54

#60 jnioche closed 2 years ago
0
java.lang.NoClassDefFoundError: io/grpc/netty/shaded/io/netty/util/concurrent/DefaultPromise$1

#59 jnioche closed 2 years ago
1
java.lang.NoClassDefFoundError: com/google/common/cache/RemovalNotification when stopping ShardedRocksDBService

#58 jnioche closed 2 years ago
1
add getInfo endpoint

#57 jnioche closed 2 months ago
1
_ShardedRocksDBService_

#56 jnioche closed 2 years ago
0
Exception caught when deleting queue

#55 jnioche closed 2 years ago
0
RocksDB backend - faster restarts

#54 jnioche closed 2 years ago
1
Faster recovery for RocksDB service implementation

#52 jnioche closed 2 years ago
0
Use Java 11

#51 jnioche closed 2 years ago
0
Expose metrics through Prometheus endpoint

#49 jnioche closed 2 years ago
0
Crawlid

#47 jnioche closed 2 years ago
0
Make logging levels configurable and be able to do it via the URLFrontier API

#44 jnioche closed 2 years ago
0
Publish service jar on Maven

#42 jnioche closed 2 years ago
0
Multithread reading from queues

#41 jnioche closed 2 years ago
0
include inactive queues in list operation

#40 jnioche closed 3 years ago
0
add getURLs functionality to command line client

#39 jnioche closed 3 years ago
0
Tuning Rockdb configuration

#38 jnioche closed 3 years ago
0
zero-padding the timestamp used in the keys for RocksDB

#37 jnioche closed 3 years ago
1
ConcurrentModificationException when calling GetStats

#36 jnioche closed 3 years ago
1
Reduce thread contention when the DB gets large

#35 jnioche closed 3 years ago
0
Odd queue names

#34 anjackson closed 3 years ago
1
GRPC error when attempting to use from Python/Scrapy

#33 anjackson closed 3 years ago
1
Stopping the Docker image with Docker kill leaves RocksDB in a corrupt state

#32 jnioche closed 3 years ago
2
List queues pagination

#31 jnioche closed 3 years ago
0
Set an explicit limit to the size of a key to 255 chars #27

#30 jnioche closed 3 years ago
0
Remove lock in putURLs for RocksDB service, implements #28

#29 jnioche closed 3 years ago
0
Remove lock in putURLs for RocksDB service

#28 jnioche closed 3 years ago
0
Set an explicit limit to the size of a key to 255 chars

#27 jnioche closed 3 years ago
0
Pagination of ListQueues

#26 jnioche closed 3 years ago
0
exception thrown when no rocksdb.path set

#25 jnioche closed 3 years ago
0
Add GetActive endpoint to API

#24 jnioche closed 3 years ago
0
Docker container running as root

#23 jnioche closed 3 years ago
0
Client needs to throttle sending seeds to Frontier

#22 jnioche closed 3 years ago
0
Add contributions policy

#21 jnioche closed 3 years ago
3
NGI0 - Updating licensing aspects according REUSE

#20 lnceballosz closed 3 years ago
3
Bump jackson-databind from 2.9.8 to 2.9.10.7 in /service

#19 dependabot[bot] closed 3 years ago
0
Rocksdb backed service implementation

#18 jnioche closed 3 years ago
0
Getting a stack dump when closing RocksDB at the end of the service

#17 jnioche closed 2 years ago
3
Scan of table failing for queues which have been completed

#16 jnioche closed 3 years ago
0
Stats for memory implementation report the wrong number of URLs in process

#15 jnioche closed 3 years ago
0
Use Picocli to configure the server

#14 jnioche closed 3 years ago
0
remove status and reorganise the API to distinguish discovered URLs from known

#9 jnioche closed 3 years ago
0
dummy server to support never refetch

#8 jnioche closed 3 years ago
0
use CLI library for parsing command line argument for Java Client

#6 jnioche closed 3 years ago
4
Java client

#5 jnioche closed 3 years ago
0