artisan-roaster-scope / artisan

artisan: visual scope for coffee roasters
https://artisan-scope.org
GNU General Public License v3.0
948 stars 257 forks source link

Please any python package to read the alog file? #1575

Closed BatbilegP closed 5 months ago

BatbilegP commented 5 months ago

I am trying to make a BI dashboards about the roasting data.

on the data architecture, im trying to eliminate the manual data exporting process by python script to read to every batch data.

I tried using the Auto-saving csv file but there isn't much of a data in it(6columns).

As right now, im exporting ranking report daily.

Is there any python package to read the alog file?

Or

Is it possible to make a feature that Auto-saving the ranking report after each batch?

Thank you

MAKOMO commented 5 months ago

I am not aware of any Python package reading Artisan produced documents. Autosaving is to save information on single roasts, but ranking reports are intended to compare several roasts thus this does not fit well. Note that Artisan profiles can also exported in JSON for simpler parsing in some languages. Profiles exported in JSON contain the same information as the native Artisan .alog format.

BatbilegP commented 5 months ago

I am working on automating the data processing to minimize human error and increase efficiency. i have thought few methods I have considered to achieve this:

Proposed Methods:

Automated Data Ingestion with Python and Airflow: There are auto-saved .alog files generated after each batch.Develop a Python script integrated with Airflow to automatically read these .alog files each day and insert the data into the database. This approach requires a Python package capable of reading .alog files.

Enhanced Autosaving Feature: Modify the existing autosaving functionality to save the complete dataset, which includes all 120 columns. This ensures all relevant data is captured and stored without the need for further processing.

Database Integration by Artisan: Since Artisan is working on the operating system, we can enable Artisan to autonomously create a database (MySQL, Postgres) and store the data locally.This local database would be directly accessible.

MAKOMO commented 5 months ago

Closing this as not planned