More obvious directory structure

AlexCLeduc commented 4 years ago

It's kind of difficult to figure out where something goes right now. I'd like to propose we reset our directory structure.

I don't have all the answers (warning: there will be conflicting ideas below). Here's a starting point for discussion:

InfoBase/
sections/
- all routes like infographic and metadata
panels/
- all the panels
- utils like declare_panels
utils/
- kitchen sink of loosely related stuff that's re-usable accross multiple top-level directories
components/
- like utils, but just react components
charts/
models/
- tables/ ?
- ensure_loaded ?
static/
- global css, svg, yaml, png
constants/ (or theme/ ?)
- colors, breakpoints, etc.

What about core/ ? explorer-common/ ? search/ ? I have 3 patterns in mind to take care of the rest

1. Using recursive utils/ and components/

To avoid src/utils getting too large, we can try and have multiple utils/ directories. If a utility becomes re-usable beyond its directory, it gets upgraded to a higher-level directory's utilities, until it ends up in src/utils/. For instance, utilities for defining new routes can be in sections/utils, because it won't be used outside sections/. Same thing goes for components.

src
- components/
- utils/
- sections/
- utils/
- components/
- panels/
- utils
- components

Since core/ has a lot of components that only get used in App.js, and a lot of bootstrapping stuff only used in src/InfoBase, most of it can be dumped into src/InfoBase/'s components and utils directories.

Another benefit of this pattern is that removing a section or panel will also remove all of its specific utils/components.

2. domain/

one problem with just having plain utils/ and components/ is that we have plenty of tightly related tools that contain both plain functions and components:

explorer_common/
search/
format

For lack of a better idea, we can nest stuff like this under src/domain/. I think we'll end up with more of these as we grow, so I'd prefer not to keep them top-level.

Note: Let's try and find a better name than domain

3. Platform/ (or foundation/)

Certain utils/components are gonna be used broadly and are more context-aware than simple functions. This could be another split point.

Think about

Analytics (e.g. log_standard_event)
in-app links (infograph_href template)
email backend
glossary tooltips/components
ensure_loaded/ graphql/ request utils
text
get_static_url

Conclusion ?

We can use any combination of recursion, domain, platform. If we use both domain and platform, we should make sure the difference is clear (e.g. should glossary/footnote utils go in domain, or platform?)

Here's a more detailed breakdown of the all the top level files (and core). Blank destination means leave it alone. If we have more opinions we can move this to a google doc or sheet. There is some stuff missing. For instance, we may decide to move some context-aware or smart components outside of components/ and into platform/ or domain/.

Current file	new destination
EstimatesComparison	sections/
FAQ	sections/
IgocExplorer	sections/
InfoBase	no change
InfoLab	sections/
PrivacyStatement	sections/
TagExplorer	sections/
TextDiff	sections/
TreeMap	sections/
about	sections/
app_bootstrap	InfoBase/
charts
common_css	static/
common_text	static/
components
contact	sections/
core/DevFip.js	InfoBase/components/
core/EasyAccess.js	InfoBase/components/
core/ErrorBoundary.js	InfoBase/components/
core/InsertRuntimeFooterLinks.js	InfoBase/components/
core/NavComponents.js (and scss)	components ? sections/components/ ?
core/Spinner.js	components/
core/Statistics.js	unsure, probably utils ?
core/SurveyPopup.js	InfoBase/components/
core/TableClass.js	tables/
core/analytics.js	utils?
core/breakpoint_defs.js	constants/
core/color_defs.js	constants/
core/color_schemes.js	constants/
core/feature_detection.js	utils ?
core/format.js	unsure should probably be split up. Also has side-effect of declaring handlebars
core/lazy_loader.js	utils? models?
core/reactAdapter.js	utils?
core/tables	move alongside all the other tables
email_backend_utils.js	utils/ ?
explorer_common	unsure
extended_bootstrap_css	static
general_utils.js	utils/
glossary	sections/
graphql_utils	utils/
handlebars	Infobase/ ? Later: consider splitting it up, making panels import their own special helpers via a `side-effects.js`
home	sections/
icons	static/ ? components/ ? constants?
infographic	sections/
link_utils.js	linking/ (or platform/)
metadata	sections/
models
panels
partition	sections/
png	static/
request_utils.js	utils/
robots	static/
rpb	sections/
search	unsure
svg	static/
tables	consider moving under models/ ?

ktw1016 commented 3 years ago

I really like the overall idea of all of this. I like the structure you laid out as starting point.

Using recursive utils/ and components/

I like this idea and we should try it out. I'm unsure how easily manageable it will be to keep separate utils and components but I think it's worth a shot

But I'm not understanding domain and platform completely. Are they basically utils and/or components but doesn't quite fit into either, so you throw them in there? Can you clarify your thoughts on the difference between the two and, (why and how) would something fit into one or the other, instead of utils or components?

AlexCLeduc commented 3 years ago

Initially I was thinking domain would be more business-related concepts and platform would be globally setup stuff like analytics, but looking through our components dir right now, I like the kitchen sink approach and don't think business-related stuff needs to be separate.

OTOH, I think there are still groups of components and utilities that we'll want to colocate, and that are often imported together. Also, some components/utils require external setup, like DocumentDescription or DocumentTitle, or modules/singletons that hold state.

If we think those criteria merit separation, we can have something like

foundation | platform | domain/

analytics
explorer_common
search
text (doesn't quite belong in models/ IMO)
nav
analytics
lazy_loader/graphql

The criteria for going in this directory can be

modules that colocate utilities and components that are often imported together
hold state in a singleton or in the module (e.g. top level let)
have dependencies on side effects (e.g. assume that the page has a #header selector)

Stephen-ONeil commented 3 years ago

Been trying to develop some thoughts on this/sketch a proposed restructuring. Heavily based on earlier discussion. Not sure I'm settled on this, but I've started to spin my tires so it's probably a good time to put it back up for discussion.

Proposal: a recursive directory structure based on four types of directories 1) utils

generic dir name
fully reusable utils (i.e. no indirect coupling to other modules, no implicit dependencies/assumptions about app state, etc)
(?) probably no sub-directories 2) components
generic dir name
fully reusable components (i.e. no indirect coupling to other modules, no implicit dependencies/assumptions about app state, etc)
(?) probably only one layer of sub-directories, to group component modules 3) platform
generic dir name
sets of utils/components/functionality, generally reusable but allowed to: 1) have strong assumptions
- e.g. assume that the page has a #header node 2) have implicit dependencies
- e.g. depend on some non-explicit one-time initialization step 2) have module level state (e.g. top level let), or use the singleton pattern
(?) could have it's own thematic sub-directories, since a "platform" could be built of many modules/sub-platforms 4) thematic sub-directories
unique, self-descriptive name
may have its own utils/components/platform/thematic sub-directories
intentionally a bit vague, sort of a know-it-when-you-see-it
- maybe because it represents some specific piece of the actual app
  - e.g. the routes dir and the contained per-route thematic dirs under it
- maybe just because it helps with organization/solidifies core concepts
  - e.g. src/charts content's could probably be tossed in the root components dir, or src/models's in platform, but both are "weighty" enough concepts to stand up on their own

Sketch of proposed structure:

/client/src/
- ./utils/
- globally reusable utils
- ./request_utils.js
- ./components/
- globally reusable components
- ./SomeComponent/
  - ./SomeComponent.js
  - ./SomeComponent.scss
- ./platform/
- sets of related utils/components/functionality, especially for code meeting any of the following criteria: 1) have strong assumptions/implicit dependencies
  - e.g. assume that the page has a #header node, depend on some non-explicit one-time initialization step, etc 2) have module level state (e.g. top level let), or use the singleton pattern 3) modules that collocate utilities and components that are often/necessarily imported together
- ./Search/
- ./analytics.js
- ./InfoBase/
- application entry point, e.g. the application bootstrapper, the React Router configuration, etc.
- ./components/
  - pre-router components like ErrorBoundary, etc.
  - or would many of those be "platform" code??
- ./index_html/
  - templates and data for the generation of index html files
- ./models/
- ./platform/
  - ./ensure_loaded.js
- ./subjects/
- ./tables/
- ./routes/
- ./platform/
  - ./StandardRouteContainer.js and other route/nav components?
- ./Infographic/
  - ./utils/
  - ./infographic_link.js
  - ./Infographic.js
  - the route module, clearly belongs at the root of this dir
  - ./PanelFilterControl.js
  - not a generally reusable component, or part of some self-contained system
  - highly coupled to ./Infographic.js itself, could be inlined in it but kept separate for organization reasons
  - so it should also live in the root of this dir, rather in a ./components sub-dir, right?
- ./panels/
- all the panels
- utils like declare_panels
- could potentially be a thematic sub-directory of the infographic route dir, but might warrant an exception because of it's size/depth (plus, technically used by two routes if we count the panel inventory, ha)
- ./charts/
- ./static/
- global css, svgs, pngs, etc
- ./theme/
- colors, breakpoints, etc.

AlexCLeduc commented 3 years ago

Seems good to me, just to make sure I understand, the root platform/ would have the explorer and search?

ktw1016 commented 3 years ago

For models/, would populate files go into platform? If it's just ensure_loaded in platform, I think ensure_loaded can just stay on the root of models/

I'd generally prefer to minimize the use of recursive subdirectories of components/, platform/, utils/, etc if it's just 1 file in there just so the whole directory structure tree doesn't become too huge.

I think it's worth keeping charts/ and panels/ at root directory for more visibility

AlexCLeduc commented 3 years ago

Agreed with @ktw1016 that charts/ panels can go in the root directory.

I disagree that single files should replace utils/components. Instead, I'd rather utils start out as utils.js modules, and get "upgraded" to utils/ directories once they get large, in a way that doesn't lead to import refactors.

Stephen-ONeil commented 3 years ago

Archived doc draft from closed PR, to revisit when the repo is less active and a full directory reorg is more feasible. https://github.com/TBS-EACPD/infobase/blob/4efbe04269749a86da404f3aa431db465369dff4/docs/client/directory-structure.md

TBS-EACPD / infobase