search

keotl / ohdieux

A podcast feed converted for Radio-Canada Ohdio / Un convertisseur de flux de balados pour Radio-Canada Ohdio
https://ohdieux.ligature.ca
GNU Affero General Public License v3.0
31 stars 5 forks source link

"Not found" error from neuro API when using human-readable programme IDs #1

Closed Tortue-qc closed 2 years ago

Tortue-qc commented 2 years ago

The application you developed is exactly what I'm looking for Unfortunately, it is broken. The generated link always shows "An error occurred while handling this request." I've tried locally and with the link in the readme.

keotl commented 2 years ago

Hi there, I'm sorry to hear about the issue. Since this app is basically reverse-engineering a connection to Ohdio's API, I have found it to be a bit picky depending on the requested programme ID. Could you provide the ID you are using?

If you are running the app locally, it would also be quite helpful to paste the stack trace here. It should be logged to the console by default, or you could alternatively add debug=True as a parameter in main.py to have the stack trace returned in your browser.

application = JivagoApplication(ohdieux, context=Context, debug=True)
Tortue-qc commented 2 years ago

Hi, thanks a lot for your time.

The ID I try to use is la-journee-est-encore-jeune I've also tried with grand-ecart-wauthier-entrainement-forme

and here's the stack from the console:

2022-09-07 19:17:46 [RssResource] [INFO]  Refreshing programme la-journee-est-encore-jeune.                                                                                                         
2022-09-07 19:17:46 [urllib3.connectionpool] [DEBUG]  Starting new HTTPS connection (1): services.radio-canada.ca:443                                                                               
2022-09-07 19:17:46 [urllib3.connectionpool] [DEBUG]  https://services.radio-canada.ca:443 "GET /neuro/sphere/v1/audio/apps/products/programmes-v2/la-journee-est-encore-jeune HTTP/1.1" 404 72     
2022-09-07 19:17:46 [OhdioApi] [DEBUG]  Failed to retrieve programme la-journee-est-encore-jeune.                                                                                                   
2022-09-07 19:17:46 [ExceptionFilter] [ERROR]  Traceback (most recent call last):                                                                                                                   
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/error_handling/unknown_exception_filter.py", line 17, in doFilter              
    chain.doFilter(request, response)                                                                                                                                                               
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter                                                        
    self.filters[0].doFilter(request, response, self.getNextChain())                                                                                                                                
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/templating/template_filter.py", line 25, in doFilter                                                      
    chain.doFilter(request, response)                                                                                                                                                               
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter                                                        
    self.filters[0].doFilter(request, response, self.getNextChain())                                                                                                                                
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/request/json_serialization_filter.py", line 20, in doFilter                                          
    chain.doFilter(request, response)                                                                                                                                                               
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter                                                        
    self.filters[0].doFilter(request, response, self.getNextChain())
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/request/http_form_deserialization_filter.py", line 16, in doFilter
    chain.doFilter(request, response)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter
    self.filters[0].doFilter(request, response, self.getNextChain())
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/streaming_response_header_filter.py", line 13, in doFilter
    chain.doFilter(request, response)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter
    self.filters[0].doFilter(request, response, self.getNextChain())
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/body_serialization_filter.py", line 20, in doFilter
    chain.doFilter(request, response)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter
    self.filters[0].doFilter(request, response, self.getNextChain())
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/error_handling/application_exception_filter.py", line 27, in doFilter
    raise e
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/error_handling/application_exception_filter.py", line 21, in doFilter
    chain.doFilter(request, response)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 28, in doFilter
    self.filters[0].doFilter(request, response, self.getNextChain())
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/system_filters/request_scope_filter.py", line 18, in doFilter
    chain.doFilter(request, response)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/filter/filter_chain.py", line 20, in doFilter
    gotten_response = invoker.invoke(request)
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/invocation/rewrite/path_rewriting_route_handler_decorator.py", line 16, in invoke
    return self.decorated.invoke(ModifiedPathFlyweightRequest(request, self.new_path))
  File "/home/christian/Documents/github/ohdieux/venv/lib/python3.10/site-packages/jivago/wsgi/invocation/resource_invoker.py", line 29, in invoke
    function_return = method(*parameters)
  File "/home/christian/Documents/github/ohdieux/ohdieux/resource/rss_resource.py", line 43, in cached
    cache_entry["content"] = self.get_manifest(programme_id, reverse)
  File "/home/christian/Documents/github/ohdieux/ohdieux/resource/rss_resource.py", line 51, in get_manifest
    {"programme": programme.programme,
  File "/home/christian/Documents/github/ohdieux/ohdieux/ohdio/ohdio_programme_response_proxy.py", line 25, in programme
    self._fetch_programme()
  File "/home/christian/Documents/github/ohdieux/ohdieux/ohdio/ohdio_programme_response_proxy.py", line 77, in _fetch_programme
    json = self._api.query_programme(self.programme_id)
  File "/home/christian/Documents/github/ohdieux/ohdieux/ohdio/ohdio_api.py", line 43, in query_programme
    raise ApiException(response.text)
ohdieux.ohdio.ohdio_api.ApiException: {
  "message": "Not found",
  "requestId": "6aQ0rcw5ULGecm4PNXuzr7"
}

2022-09-07 19:17:46 [werkzeug] [INFO]  127.0.0.1 - - [07/Sep/2022 19:17:46] "GET /rss?programme_id=la-journee-est-encore-jeune HTTP/1.1" 500 -
keotl commented 2 years ago

I just had a quick look at the requests and it seems that for many (new?) shows, the /programmes-v2 endpoint using the human-readable programme ID returns a "not found" error. I found the /programmes-without-cuesheet-v2 endpoint though that seem to work fine for all shows, assuming we use numeric programme IDs. From what I can tell, the payload format appears to be identical.

https://services.radio-canada.ca/neuro/sphere/v1/audio/apps/products/programmes-without-cuesheet-v2/672/1
(672 is for la-soiree-est-encore-jeune)

I think going forward, we should call the new endpoint and update the instructions to use the numeric ID, while keeping the previous endpoint as a fallback.

Tortue-qc commented 2 years ago

Almost working.

After changing the base_url in ohdio_api.py

From https://services.radio-canada.ca/neuro/sphere/v1/audio/apps/products/programmes-v2/ 
To https://services.radio-canada.ca/neuro/sphere/v1/audio/apps/products/programmes-without-cuesheet-v2/

I was able to have it working with 672 --> la-soiree-est-encore-jeune

But for both podcast I'm interest with, it only return the information about the podcast (title, description, image...) without the episode Ex: https://services.radio-canada.ca/neuro/sphere/v1/audio/apps/products/programmes-without-cuesheet-v2/5614/1

For reference, I've tried with

keotl commented 2 years ago

Interesting. The entities are probably slightly different from an endpoint to the next. I think the best way to proceed would be to reimplement the API calls from scratch into a OhdioReaderV2 and try calling both the new and old ones anytime a request is received. Feel fry to give it a go if you'd like! Otherwise, I'll try to work on a fix sometime during the next week.

keotl commented 2 years ago

@Tortue-qc The issue should be fixed on the website. Make sure to use the numeric programme ID from now on.