issues
search
crawler-commons
/
url-frontier
API definition, resources and reference implementation of URL Frontiers
Apache License 2.0
46
stars
12
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Added method to count URLs in a frontier
#114
klockla
opened
5 days ago
0
Use replace instead of replaceall
#113
klockla
closed
5 days ago
0
enable grpc server reflection
#112
jdpedrie
closed
3 weeks ago
5
proposed v3 api definition
#111
jdpedrie
opened
3 weeks ago
0
Update API definition to follow protobuf conventions and best practices
#110
jdpedrie
opened
3 weeks ago
4
Implement `setCrawlLimit` inside the service
#109
zaibacu
closed
1 month ago
1
Fix generation and pushing of image to Dockerhub
#108
jnioche
closed
1 month ago
1
Prerelease 2.4
#107
jnioche
closed
2 months ago
0
Reverted back to GRPC 1.66.0 and updated protoc to 3.25.5
#106
klockla
closed
2 months ago
0
Improved handling of dependencies and updated their versions
#105
jnioche
closed
2 months ago
0
Reformat some files which were not passing mvn verify
#104
klockla
closed
2 months ago
0
Added myself in the developper list
#103
klockla
closed
2 months ago
0
Better organisation of Maven plugins and update their version
#102
jnioche
closed
2 months ago
0
Added test case for discussion #94
#101
klockla
closed
2 months ago
0
Remove Ignite, implements #96
#100
jnioche
closed
2 months ago
0
Fix #87, similar behaviour re-blocking whether specitying a key or not
#99
jnioche
closed
2 months ago
0
Applied formatting + removed Ignite #96
#98
jnioche
closed
2 months ago
1
adds dependabot
#97
jnioche
closed
2 months ago
0
Remove Ignite implementation
#96
jnioche
closed
2 months ago
0
Add method ListURLs to list all URLs known in the frontier with their next fetch date
#93
klockla
closed
2 months ago
10
Add method to get URL Status (returns an URLItem)
#92
klockla
closed
2 months ago
4
Ability to set URL limit for specific domain
#91
zaibacu
closed
3 months ago
1
Ability to set Limit on domain
#90
zaibacu
closed
3 months ago
3
Bump ch.qos.logback:logback-classic from 1.4.4 to 1.4.12 in /service
#89
dependabot[bot]
closed
12 months ago
0
Fix typos
#88
MattiSG
closed
1 year ago
5
getURLs will lock whole Queue when not assign Key
#87
saselovejulie
closed
2 months ago
2
Generate multi arch Docker images
#86
jnioche
closed
1 year ago
0
Bump grpc-protobuf from 1.50.2 to 1.53.0 in /API
#85
dependabot[bot]
closed
1 year ago
0
Making the metrics variables for GetURLs protected
#84
michaeldinzinger
closed
1 year ago
1
Fix mistake for config forwarding in constructor DistributedFrontierS…
#83
michaeldinzinger
closed
1 year ago
0
Modifying functionality for rocksdb.purge to use abstract method
#82
michaeldinzinger
closed
1 year ago
1
Modifying the delete functionality for ignite.purge (Issue #74)
#81
michaeldinzinger
closed
1 year ago
2
Forwarding the config map in DistributedFrontierService constructor (Issue #73)
#80
michaeldinzinger
closed
1 year ago
3
Preventing IgniteHeartbeat race condition on the start of IgniteService (Issue #72)
#79
michaeldinzinger
closed
1 year ago
3
PutURLs calls failing with java.lang.IllegalStateException #77
#78
michaeldinzinger
closed
1 year ago
1
PutURLs calls failing with java.lang.IllegalStateException
#77
michaeldinzinger
closed
1 year ago
2
#72 #73 #74
#76
michaeldinzinger
closed
1 year ago
2
Dev work on ignite service
#75
michaeldinzinger
closed
1 year ago
0
ignite.purge answers with 'couldn't delete workdir'
#74
michaeldinzinger
closed
1 year ago
0
Configs read.thread.num and write.thread.num are ignored for IgniteService
#73
michaeldinzinger
closed
1 year ago
1
IgniteHeartbeat Thread throws a NullPointerException during start of Ignite service
#72
michaeldinzinger
closed
1 year ago
3
Some calls to GetURLs failing with java.lang.IllegalStateException
#71
anjackson
closed
2 years ago
18
Exception when trying to delete a non-existing crawl
#70
jnioche
closed
2 years ago
0
DistributedFrontierService to use threadpool for all the writes
#69
jnioche
opened
2 years ago
2
Dependency updates
#68
jnioche
closed
2 years ago
0
ShardedRocksDBService does not return ack when identical URLs are sent in short succession
#67
jnioche
closed
2 years ago
1
Multithread reading from queues
#66
jnioche
closed
2 years ago
5
Use multiple threads for putting URLs
#65
jnioche
closed
2 years ago
0
Batch write operations
#64
jnioche
closed
2 years ago
0
Use multiple threads for putting URLs
#63
jnioche
closed
2 years ago
10
Next