xrsrke / pipegoose

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
MIT License
76 stars 17 forks source link

Logger branch #35

Open KevorkSulahian opened 9 months ago

KevorkSulahian commented 9 months ago

Usage: The logger is initialized with a name and a parallel context, and it offers standard logging methods (info, warning, debug, error) with additional parameters for rank and parallel mode.

Example:

logger = DistributedLogger("example_logger", parallel_context)
logger.info("Information message", rank=0, parallel_mode=ParallelMode.GLOBAL) # Assuming rank is 0 in parallel_context
logger.warning("This is a WARNING message", parallel_mode=ParallelMode.GLOBAL)
logger.debug("This is a DEBUG message", parallel_mode=ParallelMode.GLOBAL)
logger.error("This is an ERROR message", parallel_mode=ParallelMode.GLOBAL)

The logger's behavior, such as which messages are logged and where they are saved, is determined by the rank and parallel mode passed to these methods.

KevorkSulahian commented 9 months ago

Right now the when calling the logger it's saving the messages in logs/{self.name}.txt and separating the logs with a comma

KevorkSulahian commented 9 months ago

I was able to add the unit test @xrsrke