Closed berlincount closed 4 years ago
Seems like the vbb endpoint is working now.
Can't reproduce. 😕
curl 'https://2.vbb.transport.rest/stations/900000013102/departures'
@derhuerst, try https://2.vbb.transport.rest/stations?query=oranienburg
Hm, it seems like half of the features of the API are working again.
I just discovered dfb6c4aed45b328ac5df176db27cc6859946a004. Does this mean, we will have a new working endpoint for VBB data? Are the rate-problems with VBB solved?
As mentioned above, I introduce breaking changes in hafas-client
or any of its wrapper libraries (such as vbb-hafas
or db-hafas
) every now an then. When doing this, I increase the major version number by 1.
I've recently released hafas-client@4
and – building on top of that – vbb-hafas@6
, db-hafas@4
, bvg-hafas@2
, etc. For the wrapping REST APIs, I've created a branch each for the old version (2
in vbb-rest
, 2
for db-rest
and 1
for bvg-rest
) and adapted master
to the aforementioned latest *-hafas
versions.
Long story short: 3.vbb.transport.rest
/3.db.transport.rest
/2.bvg.transport.rest
represent the latest data format based on hafas-client@4
. I haven't announced these APIs publicly, so expect sudden changes.
Does this mean, we will have a new working endpoint for VBB data?
With the new 3.vbb.transport.rest
API the fundamental problem is not solved: The VBB endpoint may still block the servers' IP address(es) if there are too many requests.
We can reduce the chance by load-balancing over multiple servers though, so please deploy the latest REST APIs on your servers or cloud services (such as DigitalOcean, Scaleway, AWS, Now or Heroku):
vbb-rest#3
/db-rest#3
/bvg-rest#2
./health
HTTP route (e.g. curl 'https://my-vbb-rest-3-instance.example.org/health
).Alright, thank you for clarifying.
I guess I was a little confused by the 3 in 3.vbb.transport.rest
I thought that you maybe had a talk with the VBB people... But yeah, your explanation makes sense :)
3. Post its URL here.
Hi, I can offer these for the lb:
https://db.fishlevel.com
https://vbb.fishlevel.com
https://bvg.fishlevel.com
The instances run on a bare-metal server located in a data center in Berlin. I could also add the version prefix in the domain if you prefer. Do you know how much traffic is generated on average per month?
@jonathan-reisdorf
Thank you! 💚
I could also add the version prefix in the domain if you prefer.
Yes please, it helps me not to misconfigure something by accident. You could adopt the naming scheme of @juliuste and me (Julius will run b.*
, I run a.*
):
c.3.vbb.fishlevel.com
-> derhuerst/vbb-rest#3
c.2.vbb.fishlevel.com
-> derhuerst/vbb-rest#2
c.2.bvg.fishlevel.com
-> derhuerst/bvg-rest#2
c.1.bvg.fishlevel.com
-> derhuerst/bvg-rest#1
c.2.db.fishlevel.com
-> derhuerst/db-rest#2
c.1.db.fishlevel.com
-> derhuerst/db-rest#1
Do you know how much traffic is generated on average per month?
All 6 endpoints together get receive ~50-500k requests per day. On my (really cheap) VPS, they usually contribute to a load of about 0.15
.
@derhuerst ok, I have set these up and also c.3.db.fishlevel.com
. The load is no problem, just the traffic might be, bc the data center where my server is housed has a fair-use policy regarding traffic (-> if it's "a couple of hundred GB" per month I might have to pay an additional fee), so let's see how much traffic is generated after a month :)
We can also define your nodes as fallback, so that they won't receive any requests if a.*
or b.*
are up.
:+1: sounds great!
@joecorcoran @deg0nz @ferby09 @k-nut @berlincount @timaschew Do you happen to have the knowledge and time to set up mirror/load balancing nodes as well? I would appreciate it.
Sorry, I don't have capacities for this right now...
I've set up 3.vbb.transport.rest
a while ago (forgot to let everyone know about this), and so far it has been running reasonably well, so please use this endpoint or 2.bvg.transport.rest
. Unfortunately both have a slightly updated output format, so make sure to check their docs.
I'm going to close this issue now, as the VBB admins haven't blocked 3.vbb.transport.rest
so far. Thanks for you patience!
I have opened a thread in the transport.rest
meta-repo to discuss hosting further: https://github.com/public-transport/transport.rest/issues/2
I have set up v5.vbb.transport.rest
, the successor of 3.vbb.transport.rest
. As usual, unfortunately the response format has changed slightly (to the format of hafas-client@5
, make sure to check its migration guide).
I will keep 3.vbb.transport.rest
running for a while, and announce its shutdown via RSS before.
thanks for having a look!