Open Ananya2001-an opened 1 year ago
Thanks, I added some suggestions to your proposal, also inline, marked with [nils].
we might have to add a dependency for the predicate option on the bbox option since there's no use of it without bbox values given
That's a very good point! Would be good to add a dependency there.
- The check_status and extend_effective_dates utils are good for the downloaded feeds
What does check_status
do exactly?
I guess the code written already is pretty good for the downloading and stuff so we will mostly copy paste
That's totally fine IMO if the code is good.
Thanks for the extra points I will keep that in mind....
What does
check_status
do exactly?
After downloading the feeds inside the output dir we can execute this file to get the status of the feeds; basically if we want to recheck...
That's totally fine IMO if the code is good.
yup :) but definitely we do need to add some more things like multi threading as u mentioned and other stuff as well....great then, I guess I can start working on it now and I will discuss things here only as we move ahead
After downloading the feeds inside the output dir we can execute this file to get the status of the feeds
ok but what is status?😅 that’s a pretty generic word. Can’t be download status. Is it just printing the schedule info or so?
status as in whether the feed is new, valid and in how many days will it get expired and stuff basically the same stuff that we show on the console when the feeds were downloaded with the fetch command for the first time here: https://github.com/azavea/gtfs-feed-fetcher/blob/6659a57fd02421f99a7fe4e01037257a80f64a4b/fetch_feeds.py#L56
they have basically given this function to kind of check the info regarding those feeds again if one wants to...
And also for the fuzzy text matching as u mentioned above do we need something like this:
Maybe typer has something like that...
I know typer has support for argument autocomplete. The kind you’re showing would also be nice to have (though it doesn’t seem fuzzy to me), but not sure if it’s easily supported. Maybe smth for later?
I meant for now, rather do like „—text-search ber“, then enter, and it prints all the datasets which have „ber“ in its name.
aah okay so basically filtering the feed urls with the search input...
Right „filter“ is a better word!
So I am laying out the changes that we would probably want to see next:
list-feeds sub command to have 3 options (--bbox, --predicate and --pretty)
*berlin*
when they want to get the exact name of a feed, but don't know what a feed is called exactly. I think fuzzy matching is over the top, a simplein
would suffice IMO. needs to be a command line arg like--search
or so. or even support regex, or both regex and simple text search. wdyt @Ananya2001-an ?fetch-feeds sub command to have one option i.e --sources
--output-dir
)…like in the azavea repo they had "gtfs"--concurrency
for a pool of threads. here we can usethreading
as it's mostly network I/O which works quite well with python's threading (actual processing wouldn't because of the GIL, which kinda makes sure python can only run one thread at a time)I guess the code written already is pretty good for the downloading and stuff so we will mostly copy paste but might have to still make some modifications as needed…feel free to add on to the list :)