search

liquidz / antq

Point out your outdated dependencies.
Other
395 stars 22 forks source link

Timeout for `antq.util.git/tags-by-ls-remote` #158

Closed vemv closed 2 years ago

vemv commented 2 years ago

Hi again,

I'm finding that antq is often taking an excessive time to complete. Whenever I hit Ctrl+\, the thread dumps tell me that it's busy with antq.util.git/tags-by-ls-remote:

"main" #1 prio=5 os_prio=31 cpu=6622.86ms elapsed=235.63s tid=0x00007fbf2880a200 nid=0x1903 in Object.wait()  [0x0000700008e74000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(java.base@16.0.1/Native Method)
    - waiting on <0x0000000419da3950> (a java.lang.ProcessImpl)
    at java.lang.Object.wait(java.base@16.0.1/Object.java:320)
    at java.lang.ProcessImpl.waitFor(java.base@16.0.1/ProcessImpl.java:433)
    - locked <0x0000000419da3950> (a java.lang.ProcessImpl)
    at clojure.java.shell$sh.invokeStatic(shell.clj:127)
    at clojure.java.shell$sh.doInvoke(shell.clj:79)
    at clojure.lang.RestFn.invoke(RestFn.java:436)
    at antq.util.git$ls_remote_STAR_.invokeStatic(git.clj:29)
    at antq.util.git$ls_remote_STAR_.invoke(git.clj:25)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.core$memoize$fn__6877.doInvoke(core.clj:6353)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at antq.util.git$tags_by_ls_remote_STAR_.invokeStatic(git.clj:47)
    at antq.util.git$tags_by_ls_remote_STAR_.invoke(git.clj:45)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.core$memoize$fn__6877.doInvoke(core.clj:6353)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at antq.diff.java$eval5361$fn__5363.invoke(java.clj:85)
    at clojure.lang.MultiFn.invoke(MultiFn.java:229)
    at antq.core$assoc_diff_url.invokeStatic(core.clj:177)
    at antq.core$assoc_diff_url.invoke(core.clj:175)
    at clojure.core$map$fn__5866.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)

Perhaps these are worth applying a timeout to?

Also, are these entries processed in parallel? The thread dump doesn't suggest so. For a large dependency tree, performing many of these sequentially could mean an excessive run time.

Cheers - V

liquidz commented 2 years ago

@vemv Thanks for your reporting!

Perhaps these are worth applying a timeout to?

I agree with applying a timeout. (It never took me that long to get there)

Also, are these entries processed in parallel?

For now, we are using pmap. https://github.com/liquidz/antq/blob/ac1f00995a1ae554305eb56266fc85c4920e000c/src/antq/core.clj#L173

liquidz commented 2 years ago

@vemv Just released v1.8.847 #167