search

webu / dalec

Django Aggregator of a Lot of External Contents (DALEC) is a generic app to aggregate contents from various sources and display it in a generic way. Designed to be customized.
BSD 3-Clause "New" or "Revised" License
4 stars 6 forks source link

readme

Django Aggregator of a Lot of External Contents aka DALEC 🤖

Stable Version Code style: black Checked with mypy Imports: isort semver Documentation Status

Django Aggregator of a Lot of External Contents (DALEC) is a generic app to aggregate contents from various external sources. Purposes are to manage (retrieve, clean, display…) those contents in a generic way independent of the source.

It's designed to be customized / extended to fit your needs.

Logo de DALEC

Tests, QA, consistency and compatibility

This app is tested to runs with:

To ensure code quality and consistency:

Concepts

Contents are categorized via :

Before you ask: yes, some contents can be duplicated (eg. an issue from the "group" channel could be duplicated in the "project" channel). It's normal and wanted. Remember : the purpose of this app is to retrieve and display the last N contents of something. If the issue we are talking about is in the last N issues of project "cybermen", it does not mean it's also in the last N issues of group "dr-who-best-friends". To manage different timelines for each channel, and keep a KISS Model, we need those duplicates.

External sources supported

External sources which could be nice to support

Installation

If you are using Django older than 3.1, the JSONField is provided from django-jsonfield-backport. For 3.1+, you MUST use a DB supporting the official django's JSONField.

You SHOULD NOT install this app but you SHOULD install one (or more) of it's children (see ). eg:

pip install dalec-gitlab dalec-nextcloud

Then, add dalec, dalec_prime and dalec's children to INSTALLED_APPS in settings.py:

# settings.py

INSTALLED_APPS = [
    # …
    "dalec",
    "dalec_prime",  # if you want to use your own Content model, don't add it
    "dalec_gitlab",
    "dalec_nextcloud",
    # …
]

And in your urls.py:

# urls.py

url_patterns = [
  ...
  re_path("^dalec/", include("dalec.urls")),
  ...
]

Usage

Each dalec's child app will probably need some specific configuration to retrieve external contents (eg: token or login/password). Please refer to this dalec's child app configuration section first.

Now your dalec's child app is configured, you can display it's X last contents somewhere in a template by using the templatetag dalec:

{% load dalec %}

{% dalec app content_type [channel=None] [channel_object=None] [template=None] [ordered_by=None] %}

real examples:

Retrieves last gitlab issues for a specific user:
{% dalec "gitlab" "issue" channel="user" channel_object="doctor-who" %}

Retrieves recent gitlab event for a group:
{% dalec "gitlab" "event" channel="group" channel_object='42' %}

Retrieves recent gitlab event for a project:
{% dalec "gitlab" "event" channel="project" channel_object='443' %}

Retrieves recent gitlab issues for a project:
{% dalec "gitlab" "issue" channel="project" channel_object='404' %}

Retrieves recent gitlab issues for multiple projects:
{% dalec "gitlab" "issue" channel="project" channel_objects='["42","443"]' %}

Retrieves recent gitlab issues for multiple projects and order them by descending
issue internal ID (default is `last_update_dt`):
{% dalec "gitlab" "issue" channel="project" channel_object='42' ordered_by="-iid" %}

dalec_example

An example app is packaged to get a working example which does not require any extra configuration.

{% load dalec %}
<h1>Last quarters (very usefull, isn't it ?)</h1>
{% dalec "example" "quarter" %}

<h1>Last updated establishments of french national education</h1>
{% dalec "example" "french_educ" %}

<h1>Last updated establishments of french national education depending of the Academy of Grenoble</h1>
{% dalec "example" "french_educ" channel="academy" channel_object="Grenoble" %}

Configuration

This app have general settings which can be erased for all of it's children and sometimes by content type.

DALEC_NB_CONTENTS_KEPT

Set the number of contents to keep by type. Oldest will be purged to keep only the last X contents of each channel and type. 0 means "no limit".

DALEC_AJAX_REFRESH

If True, when an user display a channel contents, an ajax requests is sent to refresh content. It's usefull if you do not want to use a cron task and/or need to get always the last contents.

DALEC_TTL

Number of seconds before an ajax request sends a new query to the instance providing instance.

DALEC_CONTENT_MODEL

Concrete model to use to store contents. If you do not want to use the default one, you should not add dalec.prime in INSTALLED_APPS to avoid to load a useless model and have an empty table in your data base.

DALEC_FETCH_HISTORY_MODEL

Same as DALEC_CONTENT_MODEL but for the FetchHistory model.

DALEC_CSS_FRAMEWORK

Name of the (S)CSS framework you use on your website. It changes the default templates used to display contents. Templates are priorized in this order (css_framework versions only used if you set a value to DALEC_CSS_FRAMEWORK):

Supported valued are:

Future supported values could be:

Customization

Styles

No styles are included inside dalec who must remains pure with no feelings. But some SCSS framework may be supported. Please refer to DALEC_CSS_FRAMEWORK setting.

Templates

You should always inherit from the default templates and use defined blocks to customise it.

List

Each app can have it's own "list" template but if it doesn't, a default one will be used. In priority order:

Item

Each content types can have it's own template. Those filenames will be tried:

For "gitlab" app, "issue" content type and "project" channel we will try :

Models

Model used to store contents is defined by the setting DALEC_CONTENT_MODEL which has the default value "dalec_prime.Content". If you want to use your own model, in your settings.py:

Manage a new external source

If you want to add a specific external source, you just have to extends dalec.proxy.Proxy and override it's _fetch method.

To create a dalec child (a proper way), you should create a new django app with the name pattern dalec_<yourExternalSourceUname>

NAQ (Never Asked Questions)

Why this logo is so ugly ?

Because I'm a developper

Aren't you afraid to be accused of blasphemy by the Daleks ?

A Dalek which have a concept of Blasphemy can not be a sane Dalek! https://youtu.be/6ThpkjDgdvY?t=162

What's the dalec origin ?

The Daleks are a fictional extraterrestrial race of mutants principally portrayed in the British science fiction television programme Doctor Who. Name (and logo) of this django app is directly related to them.

This project was made possible thanks to the Open Space Maker Federation whose goal is to open up the world of space infrastructure to as many people as possible.
They need boards to aggregate contents from different sources (gitlab, discourse…). We (Webu) didn't find any applications which fit our needs, so we create a new one and released it under the MIT licence.