Folder structure

[gustavwilliam]

@ReneganRonin proposed the following project structure in #1. The discussion is being migrated to this issue.

.
├── bot
│   ├── cogs
│   │   └── __init__.py
│   ├── __init__.py
│   └── __main__.py
├── LICENSE
├── Pipfile
├── Pipfile.lock
└── README.md

I don't think the root-level files matter too much in this structure. They're almost always gonna be the same ones. The part I'm personally interested in exploring is the structure inside /bot.

https://github.com/python-discord/bot/tree/master/bot has a nice and organised structure imo. I would love to implement something similar. Any thoughts on this?

[uncomfyhalomacro]

I looked at their folder structure, I do admit that it is well organized. I guess implementing it is a good idea. Should I modify it to be as similar as theirs on #1? Or should we create a separate issue? What do you think?

[gustavwilliam]

I've closed #1 now, and we can discuss this here. I don't think a PR is the right place to discuss it. Maybe we could lay out a clear diagram of what we're actually thinking, here in the issue as well, so it's not as vague as "let's do it like they did." Would you like to draft it?

[uncomfyhalomacro]

I will do it then. Anyone interested can help.

[Shivansh-007]

| - gurkbot
    | - exts
        | - moderation
            | - __init__.py
        | - fun
            | - __init__.py
        | - __init__.py
    | - utils
        | - __init__.py
    | - __init__.py
    | - __main__.py
    | - bot.py
    | - constants.py
| - .env.template
| - .flake8
| - .gitignore
| - LICENSE
| - Pipfile
| - Pipfile.lock
| - README.md

Came up with something, similar to yours

[uncomfyhalomacro]

.
├── .env
├── .flake8
├── LICENSE
├── README.md
├── __init__.py
├── __main__.py
├── bot.py
├── constants.py
├── exts
│   └── __init__.py
└── utils
    └── __init__.py

Improved visuals using tree, as we can see, bot.py is where we register or import our commands from the following directory exts/ and utils/. exts/ is where we will add our commands inside a folder called main/ and our "for fun" commands inside a folder called fun/. The utils/ folder is where we will hold our custom utilities (e.g. conversion of time, regex stuff) that can be imported for our other commands. This way our code won't be as cluttered as much as it encourages reusable code. I was also thinking of adding resources/ and api/and db/ as well but that would depend on the growing complexity of the bot.

[Shivansh-007]

Hmmm, i would like the them to be inside a gurkbot or bot folder. Like:

.
├── .env
├── .flake8
├── LICENSE
├── README.md
└── gurkbot
      ├── exts
      │   └── __init__.py
      ├── utils
      │  └── __init__.py
      ├── __init__.py
      ├── __main__.py
      ├── bot.py
      └── constants.py

[RandomDev26]

I think Having everything all the bot stuff under the bot folder would be better too

[uncomfyhalomacro]

Yeah I agree.

[gustavwilliam]

How does this compare to python-discord/bot?

[uncomfyhalomacro]

It looks similar, I know that we based it off from the python-discord/bot but the idea of our structure is to keep things separate as possible so it wont get cluttery. IMO our structure might even be better :D

[gustavwilliam]

Yeah, for sure! Do you have any concrete differences? M it’s just good to know. What’s not the same.

[v1nam]

yeah, it looks good to me, maybe add a resources folder for storing stuff like yaml files?

[RandomDev26]

I think it's pretty much ready, we just need to add the resources folder.

[gustavwilliam]

yeah, it looks good to me, maybe add a resources folder for storing stuff like yaml files?

I don't think resources would be the right place for such files. Possibly /config, but I think the Python way of storing the files actually works just fine. I'm not sure if adding it to a folder would make sm things harder to import.

[RandomDev26]

I'm not sure if adding it to a folder would make sm things harder to import.

Won't make that much of a difference

[v1nam]

There's no need of importing from the resources folder, its just for storing stuff like yaml and json, which are loaded on the working file

[uncomfyhalomacro]

config folder makes sense to store all config files i mean, hence the name.

[15696]

what about a CONTRIBUTING.md?

[15696]

i think we might need some tests as well

[Shivansh-007]

i think we might need some tests as well

Test can be thought of later, i don't think it is needed now.

[Shivansh-007]

Updated:

.
├── .env
├── .flake8
├── LICENSE
├── README.md
└── gurkbot
      ├── exts
      │   └── __init__.py
      ├── utils
      │  └── __init__.py
      ├── resources
      │  └── (empty)
      ├── __init__.py
      ├── __main__.py
      ├── bot.py
      └── constants.py

[15696]

why do exts need init.py again?

[Shivansh-007]

why do exts need init.py again?

so that we can mark it as Python package directory

[15696]

why do exts need init.py again?

so that we can mark it as Python package directory

why do we need that

[uncomfyhalomacro]

He just basically answered your question. Also how can we even import our commands if there is no __init__.py?

[fisher60]

@mudkipdev yes, the extensions/cogs need to be imported as modules from the other directories. Therefore they need an init.

[vivax3794]

@mudkipdev yes, the extensions/cogs need to be imported as modules from the other directories. Therefore they need an init.

Why would we need to import them? We are using discord.py right?

[uncomfyhalomacro]

@vivax3794 The reason why we need to break it up into separate directories is to avoid messy code organization. Writing the code for the bot inside one bot.py file is inefficient and lacks structure. Breaking it up into a folder structure where we can separate py files helps us encourage code reuse and keeps the purpose of the code of the py files isolated. Imagine writing a bot.py file and you dump all the classes, functions and commands you can think of, that is inefficient. Your code will become too large and is hard to debug -> you will scream in the depths of hell not knowing that breaking it into reusable code and breaking it into separate directories makes it easier for you and your team to debug and clean your code.

And ...that is the reason why we made this issue in the first place. Also reread this too :kissing: https://github.com/gurkult/gurkbot/issues/10#issuecomment-748415797 and https://github.com/gurkult/gurkbot/issues/10#issuecomment-748422563

[vivax3794]

What I meant is that we don't need a init file in a folder to use the cogs from it

[uncomfyhalomacro]

What I meant is that we don't need a init file in a folder to use the cogs from it

wdym we do not need? thats basically how you do it if we want to import things from different folders and we need the __init__ so we can take the imports from or to the directory. pls reread https://github.com/gurkult/gurkbot/issues/10#issuecomment-748422563

[vivax3794]

But you don't import a file to use the cogs from it?

[uncomfyhalomacro]

@vivax3794 we do. see https://github.com/python-discord/bot/tree/master/bot and look at the structure. they have an __init__.py file. It is not because we copied it, but that is how we should structure it.

[vivax3794]

Well if we want to group the cogs it into folders yes, but will the bot really be that large?

[uncomfyhalomacro]

Should we wait if it becomes large?

[uncomfyhalomacro]

Also i do think that the folder structure is the backbone of the project, and even if the bot is small, at least we have organized it so well, even new developers can navigate and read the code easily.

[gustavwilliam]

what about a CONTRIBUTING.md? i think we might need some tests as well

@mudkipdev definitely! However, let's make sure to keep this issue on topic. That can be discussed In a discussion or another issue.

[gustavwilliam]

Should we wait if it becomes large?

@vivax3794 I agree with @ReneganRonin that we should make a solid structure from the start. There's also an issue with committing, where merge conflicts will be much more common if everyone is working on the same file. It's just all around nicer to split things up into logical files and have a solid structure to start with.

And I hope that we will expand this bot a lot in the future. Making sure that things are organised from the start is key to a maintainable project.

[uncomfyhalomacro]

.
├── bot
│   ├── bot.py
│   ├── constants.py
│   ├── exts
│   │   ├── fun
│   │   │   └── __init__.py
│   │   ├── __init__.py
│   │   └── main
│   │       └── __init__.py
│   ├── __init__.py
│   ├── __main__.py
│   └── utils
│       └── __init__.py
├── config
├── LICENSE
└── README.md

or

.
├── bot
│   ├── bot.py
│   ├── constants.py
│   ├── exts
│   │   ├── fun
│   │   │   └── __init__.py
│   │   └──__init__.py
│   ├── __init__.py
│   ├── __main__.py
│   └── utils
│       └── __init__.py
├── config
├── LICENSE
└── README.md

These two are based on the collective ideas of the gurkult. If you have any comments or suggestions such as changes and stuff, better start early. Because we need to move forward for this project in the next 36 hours.

[gustavwilliam]

.
├── bot/
│   ├── bot.py
│   ├── constants.py
│   ├── exts/
│   │   ├── fun/
│   │   │   ├── ...
│   │   │   └── __init__.py
│   │   ├── ...  # Categories
│   │   └── __init__.py
│   ├── __init__.py
│   ├── __main__.py
│   └── utils/
│       ├── ...
│       └── __init__.py
├── .github/
│   ├── workflows/
│   └── ...
├── tests/
├── config/
├── .gitignore
├── LICENSE
├── CONTRIBUTING.md
└── README.md

Thanks to @ReneganRonin for the proposal. This is a slightly refined version of it, where ... represents that more files will of course exist there, but that they are specific to what we decide to add and may change often.

What is the purpose of exts/main in the proposal sent above?

[uncomfyhalomacro]

@gustavwilliam I just thought that the main commands for the bot be there but I also think that is unnecessary. Because the files directly under the ext/ folder can or will be the main commands, any other folder (not files) under ext/ will be custom commands and features (e.g. fun/ where we will put our crazy stupid commands).

[gustavwilliam]

Yeah. I think just putting it under /ext/<category> will make sense. Some "main" commands probably won't exist either. They'll all be part of some category.

[uncomfyhalomacro]

@gustavwilliam We will try to implement that kind of folder structure then. We will add some categories once we have thought what kind of category should the command be.

[gustavwilliam]

Sounds perfect! We don't need to add any categories for now, though. Maybe exts/utils, if you want to try add something. Are you opening a PR?

[uncomfyhalomacro]

@gustavwilliam so we will move utils/ under /exts/utils? i see. I will make a PR as early as 4 hours (if i am not busy)

[gustavwilliam]

Nope! The utils in bot/utils are general utils, like pagination, for the cogs. The cogs in exts/utils are util commands, like reminder and loading cogs.

[uncomfyhalomacro]

@gustavwilliam Thanks for the clarification! Now we are ready to implement it :dancers:

[Vyvy-vi]

I think this issue was closed by gurkbot#16: Folder Structure