search

eschares / APC_tracker

Automated tracker of list APCs
1 stars 1 forks source link

Implement Github action to check for (and track) daily updates #1

Open rafguns opened 1 year ago

rafguns commented 1 year ago

I've created an experimental branch in a fork to see if we can check for updates on an automated basis using GH actions. This doesn't use the comparison functionality you wrote but rather relies on git to track changes for us. For this reason, I save the Excel as CSV, so git can generate diffs etc. In addition, it uses csv-diff to generate nice commit messages that summarize what has changed.

The slightly annoying thing is that I now have to wait for actual updates before I know if it works. So consider it an experiment for now, but I thought you'd like to know :-)

eschares commented 1 year ago

Thanks! Looks like you did this on Oct 11. I have a daily data pull running and I did detect a change in Elsevier's APC list on October 17. For example, Academic Radiology (1076-6332) went from $3320 to now $3510. Overall, 65 journals changed prices, ranging from -$670 (a decrease?) to +$630, and an average of +$238.

This was all done manually in Excel :)

rafguns commented 1 year ago

Hi Eric, I found out that my GH Action didn't run daily because it was in another branch than main. I just fixed this and now it seems like it detected the change as well, see https://github.com/rafguns/APC_tracker/commit/be4781f5792c7b4d90dec14604726fafa6fecc29

(The '1 column removed' is a one-time thing, due to a minor mistake. I initially also saved the index to the CSV.)

eschares commented 1 year ago

Wow, that looks amazing! Very cool. Great summary page.

Can we add the journal name with the ISSN on those that have a change detected? Also, what is your vision for where this summary gets displayed? Right now it's a nice format, but hard to find on the repo page.

rafguns commented 1 year ago

Just to clarify; this overview is the commit message. Every change would generate a new git commit with a similar message. I agree it's maybe not the most user-friendly summary. But since every change is recorded in git, it would be possible to use something like git-history to get all changes in a format that's easier to analyze and build graphs etc. on top of that, though I admit I haven't actually built that yet :-)

I think it should be possible to include the journal name though that might require forking csv-diff (not sure).