search

rukmanigopalan / adlsguidancedoc

80 stars 30 forks source link

Data Load into Data Lake #6

Open nidhisinha opened 4 years ago

nidhisinha commented 4 years ago

Should this document also cover how data should be loaded in the data lake. Especially when direct loads from 3rd part tools and APIs like Adverity or social sources.

ADF does support many of our known Data Sources but not all and hence the challenges around interops. How can we expose ADLS as outputs?

hurtn commented 4 years ago

It's a great topic Nidhi, and the obvious pattern ADF does not cover is the streaming sources. I think there is placeholder for this to be included a later version of the document but currently only Databricks/spark can support that loading pattern. Rukmani to confirm.