search

nitishgupta / wikiparser

Wikipedia Dump Parser
7 stars 2 forks source link

Weird log files #20

Open nitishgupta opened 7 years ago

nitishgupta commented 7 years ago

  1. In the logs folder, all log filenames start with ExtractedWiki.... It is probably because you are using the WikiExtract class (or something for logging). This shouldn't happen.

  2. All log files only contain the following text: Nov 06, 2017 12:34:42 PM edu.illinois.cs.cogcomp.wikiparser.wikiparse.WikiExtractParser INFO: Static Function So there is no apparent logging happening. your code should have log messages to keep a track of what is happening.

  3. Ideally, the log files should be names ClassName_Date_Time.log

rxtan2 commented 7 years ago

Resolved: 1) names of logfiles now follow this order: ClassName_Date_Time.log 2) Added relevant log messages

nitishgupta commented 7 years ago

I feel that classes that have a main should only define a Logger object (i.e. a new log file). Currently other classes do so. For example, if the user runs JWPLParser class, there will be three log files for a single run (PageLineParser, CategoryParser, CategoryPagesParser) but no log file called JWPLParser (the original class run by the user).

Therefore, for classes that define main should start a new log file. All other classes (whose objects) are made in other class functions should use the logger from the 'parent' class. For example, PageLineParser, CategoryParser, and CategoryPagesParser classes should use the logger object defined in JWPLParser.

rxtan2 commented 7 years ago

The runner class is used to start the wikiextractparser. However, the wikiextractparser class passes a logger into the threadpoolexecutor class. Do you think that's fine or should I just instantiate the logger object in the runner class?

nitishgupta commented 7 years ago

Can you put links to the classes in your comments. It is hard to understand this way. For example, there is no 'runner' class.

rxtan2 commented 7 years ago

https://github.com/nitishgupta/wikiparser/blob/a2cd67d7a00e1e8261c965e72d6b2cf1849e85aa/src/main/java/edu/illinois/cs/cogcomp/wikiparser/wikiparse/Runner.java#L18

It calls the ExtractWiki function from the WikiExtractParser class.

https://github.com/nitishgupta/wikiparser/blob/a2cd67d7a00e1e8261c965e72d6b2cf1849e85aa/src/main/java/edu/illinois/cs/cogcomp/wikiparser/wikiparse/WikiExtractParser.java#L64

The logger object is defined in this class and passed into the threadpoolexecutor class in this line.

nitishgupta commented 7 years ago

Why don't you just make a main function in the WikiExtractParser class and get rid of the Runner class. Is there any specific reason that I might be missing?

rxtan2 commented 7 years ago

I had a main function in the WikiExtractParser initially but you said to remove that and create a Runner class to call the WikiExtractParser. I just wanted to clarify before making the changes.

nitishgupta commented 7 years ago

Okay. Sorry for the confusion. For now let's get rid of the Runner class and make a main function in the WikiExtractParser class. This should solve your doubt about the Logger, right?

rxtan2 commented 7 years ago

Yeah, no problem. I will make the changes.

rxtan2 commented 6 years ago

Resolved: https://github.com/nitishgupta/wikiparser/blob/57ba9bcc98936fb0c2f71f398e403873d7408ad6/src/main/java/edu/illinois/cs/cogcomp/wikiparser/utils/ParserLogger.java#L23