issues
search
crawler-commons
/
url-frontier
API definition, resources and reference implementation of URL Frontiers
Apache License 2.0
44
stars
11
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add method ListURLs to list all URLs known in the frontier with their next fetch date
#93
klockla
opened
2 weeks ago
0
Add method to get URL Status (returns an URLItem)
#92
klockla
opened
3 weeks ago
0
Ability to set URL limit for specific domain
#91
zaibacu
closed
3 weeks ago
1
Ability to set Limit on domain
#90
zaibacu
closed
3 weeks ago
3
Bump ch.qos.logback:logback-classic from 1.4.4 to 1.4.12 in /service
#89
dependabot[bot]
closed
8 months ago
0
Fix typos
#88
MattiSG
closed
12 months ago
5
getURLs will lock whole Queue when not assign Key
#87
saselovejulie
opened
1 year ago
1
Generate multi arch Docker images
#86
jnioche
closed
1 year ago
0
Bump grpc-protobuf from 1.50.2 to 1.53.0 in /API
#85
dependabot[bot]
closed
1 year ago
0
Making the metrics variables for GetURLs protected
#84
michaeldinzinger
closed
1 year ago
1
Fix mistake for config forwarding in constructor DistributedFrontierS…
#83
michaeldinzinger
closed
1 year ago
0
Modifying functionality for rocksdb.purge to use abstract method
#82
michaeldinzinger
closed
1 year ago
1
Modifying the delete functionality for ignite.purge (Issue #74)
#81
michaeldinzinger
closed
1 year ago
2
Forwarding the config map in DistributedFrontierService constructor (Issue #73)
#80
michaeldinzinger
closed
1 year ago
3
Preventing IgniteHeartbeat race condition on the start of IgniteService (Issue #72)
#79
michaeldinzinger
closed
1 year ago
3
PutURLs calls failing with java.lang.IllegalStateException #77
#78
michaeldinzinger
closed
1 year ago
1
PutURLs calls failing with java.lang.IllegalStateException
#77
michaeldinzinger
closed
1 year ago
2
#72 #73 #74
#76
michaeldinzinger
closed
1 year ago
2
Dev work on ignite service
#75
michaeldinzinger
closed
1 year ago
0
ignite.purge answers with 'couldn't delete workdir'
#74
michaeldinzinger
closed
1 year ago
0
Configs read.thread.num and write.thread.num are ignored for IgniteService
#73
michaeldinzinger
closed
1 year ago
1
IgniteHeartbeat Thread throws a NullPointerException during start of Ignite service
#72
michaeldinzinger
closed
1 year ago
3
Some calls to GetURLs failing with java.lang.IllegalStateException
#71
anjackson
closed
1 year ago
18
Exception when trying to delete a non-existing crawl
#70
jnioche
closed
1 year ago
0
DistributedFrontierService to use threadpool for all the writes
#69
jnioche
opened
1 year ago
2
Dependency updates
#68
jnioche
closed
1 year ago
0
ShardedRocksDBService does not return ack when identical URLs are sent in short succession
#67
jnioche
closed
1 year ago
1
Multithread reading from queues
#66
jnioche
closed
1 year ago
5
Use multiple threads for putting URLs
#65
jnioche
closed
1 year ago
0
Batch write operations
#64
jnioche
closed
2 years ago
0
Use multiple threads for putting URLs
#63
jnioche
closed
1 year ago
10
URLFrontier AckMessage does not contain IDs of ACKed URLInfo
#62
FelixEngl
closed
2 years ago
3
Update version of gRPC to 1.50.0
#61
jnioche
closed
1 year ago
0
RocksDB backend - faster restarts, implements #54
#60
jnioche
closed
2 years ago
0
java.lang.NoClassDefFoundError: io/grpc/netty/shaded/io/netty/util/concurrent/DefaultPromise$1
#59
jnioche
closed
1 year ago
1
java.lang.NoClassDefFoundError: com/google/common/cache/RemovalNotification when stopping ShardedRocksDBService
#58
jnioche
closed
1 year ago
1
add getInfo endpoint
#57
jnioche
opened
2 years ago
0
_ShardedRocksDBService_
#56
jnioche
closed
2 years ago
0
Exception caught when deleting queue
#55
jnioche
closed
2 years ago
0
RocksDB backend - faster restarts
#54
jnioche
closed
2 years ago
1
Faster recovery for RocksDB service implementation
#52
jnioche
closed
2 years ago
0
Use Java 11
#51
jnioche
closed
2 years ago
0
Expose metrics through Prometheus endpoint
#49
jnioche
closed
2 years ago
0
Crawlid
#47
jnioche
closed
2 years ago
0
Make logging levels configurable and be able to do it via the URLFrontier API
#44
jnioche
closed
2 years ago
0
Publish service jar on Maven
#42
jnioche
closed
2 years ago
0
Multithread reading from queues
#41
jnioche
closed
1 year ago
0
include inactive queues in list operation
#40
jnioche
closed
2 years ago
0
add getURLs functionality to command line client
#39
jnioche
closed
2 years ago
0
Tuning Rockdb configuration
#38
jnioche
closed
2 years ago
0
Next