Research database options for static sites

[tripledoublev]

I will add some options here as I look back at our previous discussions and do further research.

The first option I will explore will be inspired by Monument Lab's National Monument Audit.

[tripledoublev]

Monument Lab-inspired GitHub-based Project that uses Staticman for User-Generated Content

Our project structure is very similar to the Monument Lab National Audit, with the addition of user-generated content via Staticman, using GitHub as our database.

Monument Lab Model

The Monument Lab repository provides a framework for handling monument data from multiple sources, processing this data, and offering a user interface for data exploration.

Key Components:

• ./app/: Front-end files (HTML, CSS, JS, JSON). Data in ./app/data/ generated by Python scripts.
• ./config/: Contains JSON config files for data processing.
• ./data/: All data in various stages of processing.
• Python scripts for data ingestion, entity recognition, and processing.

Staticman for User-Generated Content

Staticman acts as a bridge between user submissions and our GitHub repository.

Key Features:

• Converts user submissions into data files and pushes them to GitHub/GitLab.
• Supports moderation and direct publishing options.
• Can be deployed on personal infrastructure or platforms like Heroku.

Potential Project Workflow

• Data Processing: Adapt Monument Lab's scripts for the specific needs of our monument data setup.
• Staticman Integration: Configure Staticman for handling user submissions.
• Front-end Development: Adapt Monument Lab's front-end to our context and include user submission forms.
• Continuous Integration: Every push to GitHub (including Staticman's data files) triggers an automatic deployment of the website.

Caveats

• Large file handling: Staticman is ideal for text-based submissions. For large file uploads (like images or videos), we'll need an alternative system. This could involve using a separate storage service.

[tripledoublev]

In our previous discussion, Airtable was mentionned as a option that would serve the role of a database and CMS.

Airtable-based Database Solution

Option of using Airtable as the database for our project, similar to Monument Lab, but with a focus on ease of use and integration of file attachments.

Airtable as Database

Airtable combines the simplicity of a spreadsheet with the complexity of a robust database. It allows us to organize data and store file attachments.

Key Advantages:

• User-friendly Interface: Airtable's spreadsheet-like interface makes data entry and manipulation straightforward for users with varying technical skills. This could facilitate user generated content moderation.
• File Attachments: Directly attach files like images, videos, and documents to individual records. This is particularly useful for our project, where submissions might include visual representations of monuments.

Potential Project Workflow with Airtable

• Data Setup and Organization: Configure Airtable base with tables and fields tailored to our project's requirements.
• User Submission Handling: Implement a system (possibly a front-end form) that feeds user submissions directly into Airtable. This can include textual data, gps coordinates and file attachments.
• File Storage: Store and manage large file attachments directly within Airtable, linked to the relevant data records.
• Integration with Front-End: Develop a front-end interface that communicates with Airtable to display data and receive user submissions.
• Continuous Syncing: Ensure real-time data syncing between Airtable and our front-end interface, allowing live updates.

Caveats and Considerations

• Rate Limits and API Usage
• Data Storage Limits: Airtable's attachment storage limits which might require optimization or clean-up strategies as the project grows.
• Access Management: Properly configure access permissions to secure sensitive data and control who can add or modify data.

Airtable presents a somewhat flexible and user-friendly approach to database management for the project, accommodating data and file attachments.