search

cyber-atomus / autowx

Automated reception & decoding of polar passing weather satellites
MIT License
26 stars 5 forks source link

Feature: Static web pages generation #4

Open rhaamo opened 7 years ago

rhaamo commented 7 years ago

Hi,

The refactoring was made to add some others features I wanted. I started working on static web pages generation, it's pretty simple, I don't wanted something really complicated.

The process is plugged as:

The templates use jinja2, no pull request yet I want to look if it's stable enought. Thumbnails use a dot-prefixed filename to not clog the output of the real-size generated files. Since I still don't have METEOR working it's not handled at all (the csv and generation are triggered only if matching NOAA). You can take a look of the code in the "static-web" branch of my fork, and I have my autowx running and generating stuff at https://noaa.lan.sigpipe.me/ (the link may broke for a few days maybe when I will rework my network at home).

I want to get METEOR working to add support to it, it would be better. Right now it's seems to be pretty ok, I think I will look at the "overlay map direction" to improve the rotation (upside down or not) in the single pass page, and look to make all links "relatives" and not prefixed by "/" every time (anyway the maps directory and img ones needs a virtualhost configuration, or, needs an autowx config to generate them in sub directories in the "web" one). I don't think I added an option to disable thumbnails generation, only web, I will add one to disable thumbs if wanted and if web not activated.

I choosed CSV because I wanted to just add simple content in it for generate the "pagination":

sat_name,automate_started,aos_time,los_time,max_elev,record_time
NOAA 18,1503425958.639888,1503428523,1503429459,47,936.0
NOAA 19,1503435618.841656,1503461286,1503462231,87,945.0
NOAA 15,1503466282.538971,1503468267,1503469146,34,879.0

A sqlite3 database could have also been used, with more informations to do the web pages generation better (like be able to regenerate all single_pass pages), I don't know.

Let me know what you think about that.

cyber-atomus commented 7 years ago

Wow. Pretty amazing stuff! Really, I'm amazed how much it's growing.

As for the static gallery, IMO it would be nice to include pass image too (like http://atomus.eu/passimg/NOAA%2019-20170824-1407-pass-map.png) if one chooses to generate them.

Having a sqlite3 database is a very good idea, simple yet powerful - this is what I'm currently doing with ISM band monitoring ( rtl433 json->sqlite3) for faster database querying, searching ~20MB .csv is memory hungry.

Again, I need some time to reimplement this on my station too. I have working METEOR (somehow, 'cause my LNA is pretty dead) and I can try things - I'm using medet+xvfb wine LrptImageProcessor. Currently have 32" tv connected to the station box to do stuff - aaaand it's at remote location. Can't tell when I get a new laptop do do stuff...

rhaamo commented 7 years ago

The pass map is generated from one of the others scripts ? I think I tested that, quickly. I will take a look on how to integrate that.

Yes sqlite3 would be better, I will look at that, probably split the main script in two or more files 'cause it's starting to be messy.

I think I tryied METEOR with the wine thingy but it didn't worked, either it failed or my rx is too bad, don't remember, I will probably add a LNA anyway so I will work on the METEOR part later.

Don't worry, this won't be ready before some times anyway and thanks for accepting the features !

rhaamo commented 6 years ago

Hi,

Quick update.

I switched from CSV to sqlite3, I reworked things for more "modularity", the standalone script to generate a pass track is now a module and generated inside pages.

I also migrated the update keps to python, I always forgotten to run the shell script, now it does it mostly automatically for me.

I got another time buggered with timezones and databases... sqlite3 doesn't support them so... I decided to just use pretty everywhere (except the display on the CLI output I think) UTC, at least it's consistent from filenames and display in web pages, hams use a lot UTC and in that way, no issue to "guess" which timezone is involved.

I still need to work for this issue to be ready to test, to split web pages generation into his own CLI target, to be used separately from the processing if wanted. Currently it's the same process as before, recording then generating images then generating pass track then generating single pass html and regenerating the indexes.

My AutoWX is still running at https://noaa.lan.sigpipe.me/ using the latest master from my improvements (a nice pass for example: https://noaa.lan.sigpipe.me/2017-12-05T14-09-06.html)

I still needs to get METEOR working, I haven't much tried it, I will look at medet+xvfb wine LrptImageProcessor, IIRC from previous attempts, not so easy and lots of dead links in the net.