deepmodeling / deepks-kit

a package for developing machine learning-based chemically accurate energy and density functional models
GNU Lesser General Public License v3.0
98 stars 35 forks source link

A logger solution proposal and is there any detail planning for developing deepks-kit? #1

Closed saltball closed 2 years ago

saltball commented 3 years ago

I notice deepks-kit manifesto from zhihu.com and deepmd Wechat Official Accounts, also The DeepModeling Manifesto. I hope I can do something.

This repo has few infomation about develop of deepks-kit, so is there any detail planning for deepks-kit, such as:

Actually I think dplibrary is one good platform for data sharing, so will there be any API for users to directly download the data, model from dplibrary? Or deepks-kit will use another work flow?

In readme.md, there are TODO section which mentions four items to do. I offer some suggests for the second one "Rewrite all print function using logging":

I think in here logging refers to python standard library logging. And according to the library using command line mostly, we can set logging in one log.py file. Here I have two methods:

  1. A setup function like (https://github.com/sehoonha/pydart2/blob/master/pydart2/utils/log.py);
  2. Or a class Logger from logging.logger like:
    
    import logging

from .util.config import getGlobValue # get global variable

all = [ "InfoLogger", "Logger" ]

""" Predefined Loggers for project. Usage:

from ..util.logger import InfoLogger

logger = InfoLogger(__name__,...) # all log above Info level will be logged.

"""

class BaseLogging(logging.Logger): def init(self, name, level=logging.INFO, file=None, console_on=False): """

    Parameters
    ----------
    name: str
        Name of logger
    level: int
        logging.INFO, logging.DEBUG,...
    file: str or None
        File name of log.
        If None, log to console.
    console_on: bool
        If console_on, log to console anyway.
    """
    super().__init__(name, level)

    # format set of log
    fmt = "PID:%(process)d [%(asctime)s]-[%(name)s]-[%(levelname)s]-[%(filename)s,line %(lineno)d] : %(message)s"
    formatter = logging.Formatter(fmt)

    # to file
    if file:
        file_handle = logging.FileHandler(file, encoding="utf-8")
        file_handle.setFormatter(formatter)
        self.addHandler(file_handle)
    # to console when not setting log file.
    else:
        console_handle = logging.StreamHandler()
        console_handle.setFormatter(formatter)
        self.addHandler(console_handle)

    # use console out when setting log file.
    if file:
        if console_on:
            console_handle = logging.StreamHandler()
            console_handle.setFormatter(formatter)
            self.addHandler(console_handle)

class InfoLogger(BaseLogging): """ Default Logger to console with INFO level. Set file to a real file to log in file. Parameters

name: str
    Name of logger
"""

def __init__(self, name, **kwargs):
    super(InfoLogger, self).__init__(name=name, level=logging.INFO, **kwargs)

class DebugLogger(BaseLogging): """ Default Logger to console with DEBUG level. Set file to a real file to log in file. Parameters

name: str
    Name of logger
"""

def __init__(self, name, **kwargs):
    super(DebugLogger, self).__init__(name=name, level=logging.DEBUG, **kwargs)

def getLogger(level=logging.DEBUG): """ Generate a logger with level. Parameters

level

Returns
-------
A BaseLogging class.
"""

class RunTimeLogger(BaseLogging):
    def __init__(self, name, **kwargs):
        super(RunTimeLogger, self).__init__(name=name, level=level, **kwargs)

    def isEnabledFor(self, level: int) -> bool:
        self.setLevel(level=getGlobValue("log_level"))
        self.level = getGlobValue("log_level")
        return super(RunTimeLogger, self).isEnabledFor(level)

return RunTimeLogger

Logger = getLogger(level=getGlobValue("log_level"))



I hope I can do something, while I'm not sure what deepks-kit maintainers are planning now.
Moreover, it will be conventional to have a project kanban and show what maintainers are planning.
tansongchen commented 3 years ago

Thank you for your detailed suggestion. We have made two kanbans about APIs and tests.

  1. Building a good API for cooperating with other SCF software is indeed our first priority task now. We plan to finish a reconstruction of the "iterate" module that decouples the ML-end (here, PyTorch) and the SCF-end (here, PySCF) in a few days, and we will launch a discussion in the GitHub Discussion zone about many implementation details after that;
  2. There are currently several kinds of class-based workflow definition, such as Reader, NetCorr and Penalty, although these classes may seem unclear how to extended them. We will do some developer doc to improve this.
  3. Logging is a good point and your suggestion about logging seems an elegant solution. We plan to do this after the code structure is almost settled down, and you are very welcome to contribute then.

p.s. I will rewrite this issue into several smaller ones and relate them with the kanbans tomorrow.