URLFrontier AckMessage does not contain IDs of ACKed URLInfo

[FelixEngl]

I describes the issue at https://github.com/DigitalPebble/storm-crawler/issues/981 the code revealing the problems (+ Unit-Test) can be found there:

• https://github.com/FelixEngl/storm-crawler/blob/ImprovedStatus/external/urlfrontier/src/main/java/com/digitalpebble/stormcrawler/urlfrontier/StatusUpdaterBolt.java#L178
• https://github.com/FelixEngl/storm-crawler/blob/ImprovedStatus/external/urlfrontier/src/main/java/com/digitalpebble/stormcrawler/urlfrontier/StatusUpdaterBolt.java#L362

[jnioche]

thanks @FelixEngl could we add a test to illustrate the problem within the test suite of URLFrontier, independently from StormCrawler? This would be useful for checking alternative implementations of the service

[jnioche]

https://github.com/crawler-commons/url-frontier/commit/ced0150d3a516ba8c8ad94b362fb8960ab2b35d6

[FelixEngl]

thanks @FelixEngl could we add a test to illustrate the problem within the test suite of URLFrontier, independently from StormCrawler? This would be useful for checking alternative implementations of the service

Yes, but I don't know when I can get to that. Because I don't know the Grpc-Framework very well and first have to get accustomed to it.

Another interesting test would be the transfer of metadata from discovered URLs to a crawler / other client.