airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
14.71k stars 3.78k forks source link

New Source: Sharepoint #4254

Open tokestermw opened 3 years ago

tokestermw commented 3 years ago

Tell us about the new connector you’d like to have

Describe the context around this new connector

Describe the alternative you are considering or using

igrankova commented 2 years ago

"Webhook-based? (no/partially/yes) partially: https://docs.microsoft.com/en-us/sharepoint/dev/apis/webhooks/overview-sharepoint-webhooks

Available authentication modes (A key/Oauth/other) API key: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/connect-to-sharepoint OAuth: https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/authorization-code-oauth-flow-for-sharepoint-add-ins Other: high-trust SharePoint Add-ins, Add-ins that use the cross-domain library

Has Sandbox environment? Yes: There are two types of Sandbox solutions: Code-based sandbox solutions, which contain a custom assembly in the package. Declarative sandbox solutions, which only contain XML-based configurations and related assets. https://docs.microsoft.com/en-us/sharepoint/administration/create-test-and-developer-environments

How to populate the account with data UI If your applications require more complex GET requests, POST requests or uses more advanced capabilities such as batching, you'll quickly notice that using the SPHttpClient is cumbersome and error-prone. In such cases, you should consider using an alternative such as the PnPjs library that offers you a fluent API that can be verified for correctness by TypeScript.

Available streams for sync User profiles and groups

Integration supports incremental sync? Yes: two kinds of synchronization: full and incremental. Manual sync: On the Start Profile Synchronization page, select Start Incremental Synchronization to synchronize only information that has changed since the last time that you synchronized. Maintain profile synchronization in SharePoint Server: https://docs.microsoft.com/en-us/sharepoint/administration/maintain-profile-synchronization

Other information/blockers Incremental sync additional info: https://www.simego.com/blog/2013/09/data-sync-incremental-sync-support"

igrankova commented 2 years ago

@sherifnada Vetting is completed, ready for implementation

sherifnada commented 2 years ago

thanks @igrankova !

armsepehr commented 1 year ago

@igrankova, Does the implementation of the sharepoint source connector is completed? You closed the issue.

igrankova commented 1 year ago

@armsepehr, I apologize, it was accidently closed during the migration to Zenhab.

nickblitz commented 10 months ago

Hey @igrankova , just checking if there has been updates made for this. I see it's in done status within the GL roadmap board? Thanks!

timxieICN commented 10 months ago

hi team - any update on this?

oustynova commented 10 months ago

Hi @timxieICN This Implementation is not in the roadmap right now. @bmcgilli @sherifnada tagging you also

sherifnada commented 10 months ago

@timxieICN would you be open to providing context on your usecase? This would help us build the connector in a way that makes it useful to you. Happy to continue the convo here or get on a 10m call if you like also

timxieICN commented 9 months ago

Thanks for the quick response, @oustynova @sherifnada . Happy to hop on a call to discuss this further.

TL/DR: We want to set up an ETL process between our internal Sharepoint site and our Cloud Infra (such as AWS S3, etc.). We want to have a way to sync up not only static files stored on Sharepoint (such as pdf, csv, word, etc.) but also the Sharepoint webpages. We also would like to have a nice way to trigger the ETL sync, such as on-demand trigger, running on a schedule, or event based trigger when new files arrive.

timxieICN commented 9 months ago

@oustynova @sherifnada - any thoughts? ^^