schildbach / public-transport-enabler

Unleash public transport data in your Java project.
https://groups.google.com/forum/#!forum/public-transport-enabler-discuss
GNU General Public License v3.0
380 stars 132 forks source link

NEGENTWEE: de.schildbach.pte.exception.NotFoundException: https://api.9292.nl/0.1/journeys?lang=en-GB&from=veldhoven%2Fbraak-40&to=station-eindhoven&searchType=departure&dateTime=2022-01-03T1308&sequence=1&realtime=true&before=1&after=5&interchangeTime=standard&byBus=true&byTrain=true&bySubway=true&byTram=true&byFerry=true #438

Open icewind1991 opened 2 years ago

icewind1991 commented 2 years ago

Query

Error

de.schildbach.pte.exception.NotFoundException: https://api.9292.nl/0.1/journeys?lang=en-GB&from=veldhoven%2Fbraak-40&to=station-eindhoven&searchType=departure&dateTime=2022-01-03T1308&sequence=1&realtime=true&before=1&after=5&interchangeTime=standard&byBus=true&byTrain=true&bySubway=true&byTram=true&byFerry=true
de.schildbach.pte.util.HttpClient.getInputStream(HttpClient.java:286)
de.schildbach.pte.util.HttpClient.get(HttpClient.java:205)
de.schildbach.pte.util.HttpClient.get(HttpClient.java:198)

Additional information

All queries for the backend seem to be broken

fuck-shithub commented 2 years ago

Can reproduce

realitygaps commented 2 years ago

can confirm. seems url being queried returns 404 on api.9292.nl so likely they changed something in the api endpoint location.

efb4f5ff-1298-471a-8973-3d47447115dc commented 2 years ago

it sucks that 9292 and NS are not working while using the app

matthijskooijman commented 2 years ago

@full-duplex When you implemented 9292 support in #129, did you have contact with 9292 about the use of their API? Or did you just reverse engineer the API from their own website or so?

I'm asking because it seems the current implementation no longer works (presumably the API was changed server-side), which is the subject of this issue.

Looking around the web, I see that 9292 offers a "business API", with public documentation (though that assumes access to swagger-generated documentation through the API itself), but it seems that needs an access token that must be requested (and maybe paid for?) and kept private (which seems impossible when embedding into an app, especially if the source is open).

matthijskooijman commented 2 years ago

Ah, also found the swagger documentation at https://reisadvies-api.9292.nl/index.html (the swagger url was documented for the "locaties-api", so I simply replaced that with "reisadvies-api" and that seems to work).

full-duplex commented 2 years ago

@matthijskooijman I've had no contact with 9292 and I very much doubt they will be helpful for this.

I do know their internal api was moved to https://wrapper.9292.nl/2.1/ and is different from both the old api and their business api, so that documentation won't be helpful. It also requires some specific headers to be set for authentication, so it's not as open as the old api.

Short term it might be easier to implement the api used at https://www.ns.nl/reisplanner/ than to fix this provider.

nicorikken commented 2 years ago

As the main reason for existance of 9292 is to provide travel information, I like to believe they might be willing to sponsor API access. I'm happy to step in and help contact 9292, but I guess that would have to be @schildbach and might even differ per application (e.g. Transport) using this code.

Because als public transport organizations contribute their data to 9292 I think this is the main organization to interface with. I know the NS API exists as well, but it should be a subset of the data 9292 can provide.

The request form for business type of API access asks for contact details for invoices, but no where a rate it mentioned. So I think it is fair to get in touch and see what can be arranged.

@schildbach and @grote are you ok if I start contacting 9292 with the intention to get business API access for Transportr?

nicorikken commented 2 years ago

I think the app developers/maintainers should be the one making the request and staying in touch with 9292. Here is the English page with more information about the API service and how to get in touch: https://9292.nl/en/business/9292-reisadvies-api

fuck-shithub commented 1 year ago

I have found a workaround. If you use trains only, you can get away with using the Deutsche Bahn API, which also shows Dutch trains. Sadly busses don't work with DB though, and you still need to visit the annoying 9292 site for that.

saqeria commented 1 year ago

@schildbach Has this issue been forgotten or is it a low priority one? I would love to finally be able to (once again) use the 'Transportr' app to manage my public transportation trips. If needed, I am also willing to contact 9292.nl to request a new API key or at least get some information about whether they are willing to provide any kind of help.

fuck-shithub commented 1 year ago

I also saw rijdendetreinen.nl is a pretty good website except the JavaScript is annoying. I wonder, if they have an API, maybe we could consider integrating it in Transportr alongside or instead of the (broken) NS and 9292

Muspah commented 1 year ago

Rijdendetreinen listens to pushdata from NDOVloket.nl, their API can be self-hosted: https://github.com/rijdendetreinen/gotrain

TheRealRWJ commented 9 months ago

Maybe This can help

I found http://openov.nl/ An opensource public transport data source Unfortunately I cant code And again Unfortunately: As far as I know the site is only available in the Dutch language

Translated statement:

freely from A to B

We work together to make high-quality planned and current travel information publicly available, freely accessible to everyone.

guushoekman commented 9 months ago

I think the easiest solution is to use the NS API: https://apiportal.ns.nl/

Even though the NS is the Dutch Railways, their API contains all modes of public transport for the whole country so implementing that would provide coverage for the whole country.

Their API is the data source for their own journey planner, which has loads of information from I think all providers. Not sure if it has all info for all providers, but if I now do a search from "Delfzijl, Borgweg" to "A.H. Homanstraat, Ten Boer" (two places in Groningen), it shows me the 20:32 Qbuzz bus is delayed by 2 minutes, it shows the price, the walking route, etc.

Implementing the NS API would mean we wouldn't have to implement any other provider (like 9292 or others suggested here), so I think this is the way to go.

skinkie commented 9 months ago

What you should do is host an OpenTripPlanner or 1313 instance yourself and run it from the available open data.

JerukPurut404 commented 8 months ago

I think the app developers/maintainers should be the one making the request and staying in touch with 9292. Here is the English page with more information about the API service and how to get in touch: https://9292.nl/en/business/9292-reisadvies-api

9292 requires its KvK number, also known as a company number, to demonstrate that it's a licensed company. At one point, I worked on a project to create an alternative frontend for 9292 using their testing API endpoint, which unfortunately wasn't very secure and I had access to it. However, I reported a bug related to API errors, and they patched it at the same time that they restricted unauthorized access to that API. I'm now planning to scrape their website and build my own API wrapper based on the official 9292 site.

skinkie commented 8 months ago

@JerukPurut404 What is the main reason you don't want to use an open source journey planner, instead of piggybacking on proprietary solutions? You make us look like fools.

JerukPurut404 commented 8 months ago

@skinkie Unfortunately, Transportr has discontinued support for 9292, which is a major drawback as my travels often involve a combination of bus and metro services. The NS API, while useful for train journeys, does not provide a bus planner or bus station information. As a result, I find myself in a predicament as I'm unable to host OpenTripPlanner or 1313 instances due to the complexity involved and the lack of resources at my disposal. I've explored an alternative solution, which is scraping webpages for the necessary information. While I acknowledge that this method may not be perfect, I believe it can still be a significant improvement over my current options. And as a side note, I certainly don't want to make anyone look like a fool

skinkie commented 8 months ago

I find it as unsustainable to scrape webpages, essentially removing ads, and present your solution as your own "free" app with your "own" journey planning results. All the open data and open source software is available to host your own service. I find this free loading behaviour displayed here morally objectionable, and a slap in the face for 10 years of open real time transport data availability in The Netherlands under the most liberal licence possible: CC-0.

JerukPurut404 commented 8 months ago

@skinkie I appreciate your perspective on this matter and I understand your concerns. However, I'd like to clarify my intentions and provide some insights into my approach.

In using scraping methods to collect information from 9292 and presenting it in a more accessible format, I want to emphasize that I do not claim ownership or authorship of the data itself. I understand the importance of giving proper credit to the original sources of the data, and I strive to provide clear attributions whenever appropriate.

Regarding ads, the situation is similar to the YouTube ad-wars. Many individuals will continue to search for methods to remove ads, whether by using alternative frontends or adblocker plugins. Thus, regardless of whether it is from me or someone else, ads are essentially removable. So By removing ads, i strive to create a clean and enjoyable platform for my users.

While we may differ in our moral perspectives, I want to emphasize that my intention is to provide a valuable service to users, using available resources in the most openly and transparent manner as possible. I want to reassure you that my actions are not intended to disrespect anyone's hard work or effort.