jina-ai / finetuner

:dart: Task-oriented embedding tuning for BERT, CLIP, etc.
https://finetuner.jina.ai
Apache License 2.0
1.47k stars 67 forks source link

Run cc loader on Databricks #767

Closed Tanguyabel closed 1 year ago

Tanguyabel commented 1 year ago

Once we have a notebook ready to load data from CC, move it onto Databricks and try to run it there. One key point is to understand how we can mount / store temporary data to be accessed by the notebook.

makram93 commented 1 year ago

Closing this ticket and moving the details to another ticket in the cc-pipeline repo here