Open Elfe-w opened 9 months ago
processed about 1300 files
Is this 1300 dot files or only parsed 1300 Java files?
收到。
So I would imagine the parsing shouldn't take that long, but your dot file generation is in serial. You also do multiple method look-ups which is a bottleneck - cpg.method
uses cpg.method.name
which is regex-based and cpg.method.nameExact
uses an index lookup which is much faster, but in any case you already have all the methods.
You could fix this, be null-safe (with .headOption
) and add some simple concurrency with (note, this is untested):
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future}
def generateDotFile(outPath: String, method: Method): Future[Unit] = Future {
method.dotCpg14.headOption.foreach(_ #> outPath)
}
@main def exec(cpgFile: String, outFile: String): Unit = {
importCpg(cpgFile)
val methods = cpg.method.isExternal(false).l
val futures = for { m <- methods } yield generateDotFile(s"$outFile${java.io.File.separator}${m.name}.dot", m)
Await.result(Future.sequence(futures), Duration.Inf)
}
Hi, I have a large number of Java files to process. In fact, I ran it for 17 hours and processed about 1300 files. Is this a normal processing speed? Am I using the wrong command? Is there any way to increase the parsing speed? Looking forward to your answer. I am using the following command
The content of the script file is as follows: