issues
search
crawler-commons
/
url-frontier
API definition, resources and reference implementation of URL Frontiers
Apache License 2.0
45
stars
12
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
enable grpc server reflection
#112
jdpedrie
closed
4 days ago
5
proposed v3 api definition
#111
jdpedrie
opened
6 days ago
0
Update API definition to follow protobuf conventions and best practices
#110
jdpedrie
opened
1 week ago
4
Implement `setCrawlLimit` inside the service
#109
zaibacu
closed
2 weeks ago
1
Fix generation and pushing of image to Dockerhub
#108
jnioche
closed
1 month ago
1
Prerelease 2.4
#107
jnioche
closed
1 month ago
0
Reverted back to GRPC 1.66.0 and updated protoc to 3.25.5
#106
klockla
closed
1 month ago
0
Improved handling of dependencies and updated their versions
#105
jnioche
closed
1 month ago
0
Reformat some files which were not passing mvn verify
#104
klockla
closed
1 month ago
0
Added myself in the developper list
#103
klockla
closed
1 month ago
0
Better organisation of Maven plugins and update their version
#102
jnioche
closed
1 month ago
0
Added test case for discussion #94
#101
klockla
closed
1 month ago
0
Remove Ignite, implements #96
#100
jnioche
closed
1 month ago
0
Fix #87, similar behaviour re-blocking whether specitying a key or not
#99
jnioche
closed
1 month ago
0
Applied formatting + removed Ignite #96
#98
jnioche
closed
1 month ago
1
adds dependabot
#97
jnioche
closed
1 month ago
0
Remove Ignite implementation
#96
jnioche
closed
1 month ago
0
Add method ListURLs to list all URLs known in the frontier with their next fetch date
#93
klockla
closed
1 month ago
10
Add method to get URL Status (returns an URLItem)
#92
klockla
closed
2 months ago
4
Ability to set URL limit for specific domain
#91
zaibacu
closed
3 months ago
1
Ability to set Limit on domain
#90
zaibacu
closed
3 months ago
3
Bump ch.qos.logback:logback-classic from 1.4.4 to 1.4.12 in /service
#89
dependabot[bot]
closed
11 months ago
0
Fix typos
#88
MattiSG
closed
1 year ago
5
getURLs will lock whole Queue when not assign Key
#87
saselovejulie
closed
1 month ago
2
Generate multi arch Docker images
#86
jnioche
closed
1 year ago
0
Bump grpc-protobuf from 1.50.2 to 1.53.0 in /API
#85
dependabot[bot]
closed
1 year ago
0
Making the metrics variables for GetURLs protected
#84
michaeldinzinger
closed
1 year ago
1
Fix mistake for config forwarding in constructor DistributedFrontierS…
#83
michaeldinzinger
closed
1 year ago
0
Modifying functionality for rocksdb.purge to use abstract method
#82
michaeldinzinger
closed
1 year ago
1
Modifying the delete functionality for ignite.purge (Issue #74)
#81
michaeldinzinger
closed
1 year ago
2
Forwarding the config map in DistributedFrontierService constructor (Issue #73)
#80
michaeldinzinger
closed
1 year ago
3
Preventing IgniteHeartbeat race condition on the start of IgniteService (Issue #72)
#79
michaeldinzinger
closed
1 year ago
3
PutURLs calls failing with java.lang.IllegalStateException #77
#78
michaeldinzinger
closed
1 year ago
1
PutURLs calls failing with java.lang.IllegalStateException
#77
michaeldinzinger
closed
1 year ago
2
#72 #73 #74
#76
michaeldinzinger
closed
1 year ago
2
Dev work on ignite service
#75
michaeldinzinger
closed
1 year ago
0
ignite.purge answers with 'couldn't delete workdir'
#74
michaeldinzinger
closed
1 year ago
0
Configs read.thread.num and write.thread.num are ignored for IgniteService
#73
michaeldinzinger
closed
1 year ago
1
IgniteHeartbeat Thread throws a NullPointerException during start of Ignite service
#72
michaeldinzinger
closed
1 year ago
3
Some calls to GetURLs failing with java.lang.IllegalStateException
#71
anjackson
closed
1 year ago
18
Exception when trying to delete a non-existing crawl
#70
jnioche
closed
2 years ago
0
DistributedFrontierService to use threadpool for all the writes
#69
jnioche
opened
2 years ago
2
Dependency updates
#68
jnioche
closed
2 years ago
0
ShardedRocksDBService does not return ack when identical URLs are sent in short succession
#67
jnioche
closed
2 years ago
1
Multithread reading from queues
#66
jnioche
closed
2 years ago
5
Use multiple threads for putting URLs
#65
jnioche
closed
2 years ago
0
Batch write operations
#64
jnioche
closed
2 years ago
0
Use multiple threads for putting URLs
#63
jnioche
closed
2 years ago
10
URLFrontier AckMessage does not contain IDs of ACKed URLInfo
#62
FelixEngl
closed
2 years ago
3
Update version of gRPC to 1.50.0
#61
jnioche
closed
2 years ago
0
Next