crawler-commons / url-frontier

API definition, resources and reference implementation of URL Frontiers
Apache License 2.0
44 stars 11 forks source link

IgniteHeartbeat Thread throws a NullPointerException during start of Ignite service #72

Closed michaeldinzinger closed 1 year ago

michaeldinzinger commented 1 year ago

Hi all, thanks for your great work! It appeared to me that the IgniteHeartbeat thread is killed due to a NullPointerException as soon as the IgniteService is started. This happens due to a race condition during the instantiation of the frontier service (class URLFrontierServer line 172). At the end of the instantiation, the IgniteHeartbeat Thread is started (class IgniteService line 392). In case the hostname and the port is not set yet (class URLFrontierServer line 179) when the IgniteHeartbeat Thread is started and calls the sendHeartbeat() method for the first time, the thread throws a NullPointerException.

[16:05:36] Ignite node started OK (id=3d3198aa)
[16:05:36] >>> Ignite cluster is in INACTIVE state (limited functionality available). Use control.(sh|bat) script or IgniteCluster.state(ClusterState.ACTIVE) to change the state.
[16:05:36] Topology snapshot [ver=1, locNode=3d3198aa, servers=1, clients=0, state=INACTIVE, CPUs=12, offheap=6.3GB, heap=2.0GB]
16:05:36.916 [main] INFO  c.u.service.ignite.IgniteService - Ignite loaded in 3371 msec
16:05:36.918 [main] INFO  c.u.service.ignite.IgniteService - Scanning tables to rebuild queues... (can take a long time)
16:05:36.919 [main] INFO  c.u.service.ignite.IgniteService - 0 queues discovered in 3 msec
Exception in thread "IgniteHeartbeat" java.lang.NullPointerException: Ouch! Argument cannot be null: key
    at org.apache.ignite.internal.util.GridArgumentCheck.notNull(GridArgumentCheck.java:49)
    at org.apache.ignite.internal.util.GridArgumentCheck.notNull(GridArgumentCheck.java:61)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2485)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2466)
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1332)
    at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:867)
    at crawlercommons.urlfrontier.service.ignite.IgniteHeartbeat.sendHeartBeat(IgniteHeartbeat.java:40)
    at crawlercommons.urlfrontier.service.cluster.Hearbeat.run(Hearbeat.java:75)
16:05:37.123 [IgniteQueueChecker] INFO  c.u.service.ignite.IgniteService - Found 0 queues, removed 0, total 0 in 8
16:05:37.470 [main] INFO  c.u.service.URLFrontierServer - Started URLFrontierServer [IgniteService] on port 7071 as localhost:7071
jnioche commented 1 year ago

thanks for reporting this @michaeldinzinger any chance you could submit a PR to fix the issue?

michaeldinzinger commented 1 year ago

Yes sure:) I created a pull request #75 which also tackles the issues #73 and #74 (in case they are real issues)

michaeldinzinger commented 1 year ago

I created another one #76 to make it DCO compliant