ericberman / MyFlightbookWeb

The website and service for MyFlightbook
49 stars 18 forks source link

For fun: Put a graph of daily flights taken (and pilots flying) over time on the home page #553

Closed KayRJay closed 3 years ago

KayRJay commented 4 years ago

In these crazy times, it would be very interesting to see how flying activity has changed over time. A graph on the home page of the number of flights taken each day over a variety of periods on the home page would cause me to visit the home page more often.

You could also chart the number of pilots reporting flights each day. Both metrics could be on one chart (using the left and right vertical axes).

The chart could appear as a selectable alternative to, or above or below the image that you display.

The standard stock price chart would be a good model:

Screen Shot 2020-03-23 at 11 03 56 AM

I'm assuming you're cooped up too, with time on your hands, looking for things to do or distract you, but hopefully healthy! 😀😷

ericberman commented 4 years ago

Interesting suggestion. I like the pictures there, since they change, so I wonder if it doesn't work best to create an image and put that in the slideshow...

KayRJay commented 4 years ago

An image wouldn't be interactive. It would be fun and interesting to be able to choose a recent period (the last n days/months, etc. as in the stock chart), or choose an arbitrary period (like Sept 1-30, 2001).

A chart like the could also be used to show the number of landings at a given airport.

Maybe, if and when you get to publishing more info on airports (issue #448), you'll want a page for things like this.

ericberman commented 4 years ago

Right, but an interactive chart means separate real estate - either below the fold, or pushing the images down. A static image can share real estate.

I'm also a bit concerned about offering that sort of detailed information. I show aggregate numbers today, that's also not accidental. A static image would let me pick the level of aggregation I'm willing to share.

ericberman commented 4 years ago

See Issue #445 for my hesitation about getting too granular on this data.

KayRJay commented 4 years ago

It would be a real shame to do this particular chart with no interactivity. That's part of the fun of this whole thing.

I mostly understand your concerns about confidentiality especially of the number of users. What other things would you like to keep confidential?

You could limit date ranges, whether trailing ranges like on the stock chart or interactive with a user-provided date range. You might require ranges to be nothing shorter than a week (or a month), or to not cover a period more than a year ago. You could limit the degree of aggregation, without making the chart really static.

Additionally, the chart is really only useful to show trends, not actual numbers. There need be no scale on the y-axis, and you wouldn't have to pop up details for a given period (like you do in Analytics). So, the data would only be visual, and still would be aggregates.

You could limit the number of different charts a user could request per week, or have a limit of 10 charts an hour or something like that.

All of these things would restrict the amount of information a competitor could get about your user base. I'm not sure there really would be much risk of leaking confidential information.

Perhaps this chart could be placed in an entirely new area on the site. For example, I could imagine a new top-level menu button ("FUN STUFF") that goes to a page with

By the way (and I'm sure it's obvious), you could compute almost all this info once/day as part of your nightly batch job. The only one that would require a live query would be the interactive chart.

I suppose you could even compute a set of charts once/day, and allow the user to choose among them. There could be a set of date ranges you supply, rather than allowing the user to specify arbitrary ranges. We'd not even know if they were truly interactive. This is also a good way to limit the disclosure of confidential information.

ericberman commented 3 years ago

Finally getting around to looking at this.

I really only have two sets of date-stamped data that could be used for graphs: users and flights. (Date of flight, not date of entry of the flight). As previously stated, I'm reluctant to publicly expose my user numbers. Flights are large enough (over 10M now!) that less reluctant to share, but I don't know that it's, well, all that interesting in an interactive graph (at least not to justify real-estate on the home page.) I actually have both graphs on my Admin page (along with a few others), which get emailed to me every day, but the graph always looks the same: it's basically an exponential - long tail to the left, with accelerating upward slope on the right. But that doesn't actually tell you anything; it's not a reflection of flight activity for at least two reasons: (a) stuff on the left represents an undersample of flights that are still on paper or in 3rd party logbooks, and (b) the acceleration on the right is as much a function of new pilots joining the system as it is new flights being entered.

What turns out to actually be interesting and revealing of trends over time - and actually isn't static - is to have a rolling window and compute not a graph of the trends but what's happening within that rolling window right now.

I'm using a 7 day window, which is large enough to have meaningful data but not so large that it's overwhelming to compute every hour or so (I'm using a 60min window, so recomputing it every hour or so).

I've enhanced the stats on the home page just a bit - I think the bulleted data is the right level here. The About page now has two lists: popular airports (with a bunch of stats per airport) and popular models. I'll address the airport stuff in my comments for #448.

As for flight activity: I'm showing # of flights, # of pilots, and breaking that down for top airports within the last 7 days. I actually am computing it for every airport within the last 7 days, but I'm not sure where/how is the relevant place to surface that.

I have no data about military vs. agriculture vs. airline vs. aerobatic flight types, so I can't display anything on this. Likewise, I h ave absolutely no idea about who the pilots are and only a guess as to the ratings they hold (which I don't store), country they live in, etc.

I already do show lots of information about aircraft - for each aircraft in your account, you can already see # of flights, how many other pilots fly that plane with you, etc. The summary I have of top models captures ICAO code, which groups similar makes/models. If I didn't do that, things would be far too granular. E.g., there are C-172R's, C-172S's, and C-172S/G1000's in the system, but these are all "C172". Steam vs. glass can be at the model level, but it can also be at the aircraft level. I suppose I could report that, but I'm not sure it's all that interesting or meaningful.

With the latest check-in, I'm showing the Longest flight in the last 7 days (total route distance) and the Furthest flight (what went furthest from departure); I've already discovered that this can be quite noisy due to people logging things that map to navaids in the system or using private names that map to real airports. E.g., it's not all that uncommon for people to put "KABC-LCL-KABC" for local flights ("LCL") that never really left the area of KABC. But alas, LCL is a legitimate airport code for an airport in Cuba.

Social networking tools is a whole separate issue, and a fairly low priority - at this point, it's a solution in search of a problem, so I haven't spent any cycles on this; either way, it would properly have its own issue to track.

So for now (when I take this live), there's a little bit of new stuff on the home page and more on the "About" page. I don't know that this needs its own page, at least not at this point.

ericberman commented 3 years ago

So I think at the moment I've done what is reasonable to do on this issue. If there are other interesting stats to propose, let's do those in individual github issues.