Structuring Python Projects

[harshraj22]

Add a wiki page on structuring Python Projects. Adding details like limiting number of .py files in the top directory, separating concerns across directories, preferring to change python search path to import from sibling directories etc

references: [1], [2]

[harshraj22]

Imports

https://github.com/harshraj22/rl_lab/blob/20527d8734c02e2a1059cf9aa7cfab6d6ceab860/lab4/models/q_learning.py#L5

Instead of using sys.path.append or modifying PYTHONPATH to include files from other directory, one should prefer using setup.py: python3 setup.py develop

References: 1 2 3 4

To uninstall: python3 setup.py develop --uninstall   1

---

Use:

from setuptools import setup, find_packages

setup(package_dir={'': 'src'})

in case of directory structure like:

.
├── setup.py
├── src
│   └── package
│       └── utils.py
│       └── __init__.py
└── tests
    └── test_module.py

and the test_module.py can contain imports like:

from package.utils import utility_func

[harshraj22]

Configs and Command Line Args

Tools like Hydra and Omegaconf make it clean and easy to manage configurations compared to Argparse (which makes code hard to read)

Here's some details on how to enforce type checking with Omegaconf

[harshraj-wadhwani]

Another good coding standards guide

the hitchhiker

[harshraj-wadhwani]

Logging

Use a logging configuration to set a system wide logging instead of repeating logger's code in each file

reference, 1

A good answer to using multiple loggers across files here

pycon talk about logging

[harshraj22]

Testing and OOPs principle

Linkedin Post

[harshraj-wadhwani]

Meta Classes

Could have used meta class to enforce the existance of mode property in the subclasses, rather than having a default on in the base class. The current used approach violates design principle.

[harshraj22]

Pre-commit hooks

Medium Article