codefortulsa / courtbot-library

Generic Court Scraper for Courbot system use
3 stars 6 forks source link

find() needs to express configuration in re storage of data #11

Open rkiddy opened 3 years ago

rkiddy commented 3 years ago

I cannot say yet that I understand how courtbot-python/oscn/courtbot-library is configuring its storage.

I am implementing something very simple in the CA code and I suggest it would be useful for others.

Right now, the CaseList.find() method seems to take some parameters. I cannot tell what they are right now because I cannot see that they are declared anywhere. Anyway. I saw, somewhere, that it might be taking a year, a name and a case number.

What I have takes a time period (which is a year, or a month, or a week, or a day) (required), a name (optional), and a case number (required).

I am envisioning, later, that we may want to search by party names but that is not for this implementation.

I also have a "fetch" parameter, which can have one of 4 values, equivalent to:

1) "force a fetch" - If called with this value, the data is fetched from the court site and is stored and then returned. It overwrites any existing stored data for the date(s).

2) "fetch if needed" - If called with this value, the data is pulled from storage. If it is not there, the data is fetched from the court site, saved and returned.

3) "do not store" - If called with this value, the data is pulled from the court site and nothing is stored.

4) "do not fetch" - If called with this value, the data is pulled from storage and no fetching is done.

I am assuming a few things.

First, there will be some way to, dynamically, look at a configuration file of some kind and use this value for the fetch parameter. If the state of the court site or the storage changes, someone may want to change this value in a running app.

Second, there will be a utility method like fetch_for_all(start_date, end_date), which will fetch and store data for the dates given.

rkiddy commented 3 years ago

I am trying to think how I can put in a PR myself to show how this might work. But "def find(self, **kwargs):" What is that? What does it declare? :--)

Sorry just not familiar with this style of python coding....