search

elastic / connectors

Source code for all Elastic connectors, developed by the Search team at Elastic, and home of our Python connector development framework
https://www.elastic.co/guide/en/enterprise-search/master/index.html
Other
59 stars 118 forks source link

[Sharepoint Online] Sharepoint REST APIs returning XML content are not handled properly #2327

Open artem-shelkovnikov opened 3 months ago

artem-shelkovnikov commented 3 months ago

Bug Description

In certain cases (not yet fully clear in which ones), Sharepoint REST API inside Sharepoint Online connectors return XML objects instead of JSON.

We need to either parse XML (this is a worse option) or force the server to always return JSON (this is a better one).

Googling "sharepoint rest api returns xml instead of json" shows that people also face same issue from time to time, for example:

https://github.com/SharePoint/sp-dev-docs/issues/1892 https://sharepoint.stackexchange.com/questions/263014/sharepoint-online-rest-api-is-returning-xml-instead-of-json https://sharepoint.stackexchange.com/questions/85325/return-json-rather-than-xml-in-a-sharepoint-2013-app-jsom

To Reproduce

Steps to reproduce the behavior:

  1. Not fully clear at this moment, unfortunately

Expected behavior

Sharepoint Server REST API always returns JSON responses when accessing content.

seanstory commented 3 months ago

It looks like we're already specifying content-type: application/json when forming the client session: https://github.com/elastic/connectors/blob/01d30bbcbeb1c4151ceb44d46cc008e8df0dc2e6/connectors/sources/sharepoint_online.py#L528-L533

IDK what else we can do.

artem-shelkovnikov commented 3 months ago

Yeah it should work, but Sharepoint REST API is a little unpredictable and what's worse we don't have a way to reproduce the issue yet. Maybe it's the odata header that does something, maybe something else?