search

jupyter / newsletter

A repo for collecting content for the Jupyter Newsletter
BSD 3-Clause "New" or "Revised" License
13 stars 13 forks source link

Newsletter Story - Jupyter + Pachyderm #57

Closed dwhitena closed 7 years ago

dwhitena commented 7 years ago

Issue category

Story suggestions

Topic:

Jupyter and Pachyderm

Headline:

Pachyderm's newest release allows Jupyter users to collaborate on versioned data and debug distributed data pipelines.

Summary of story:

The latest release, 1.3, of the open source system Pachyderm enables users to attach Jupyter notebooks to versioned data that is part of distributed data pipelines. This allows users to deploy distributed data processing in the form of DAG pipelines and, with Jupyter, interactively debug unexpected behavior, develop new pipeline stages, and visualize data at any point in the DAG. It also provides a chronologically ordered record of data (because Pachyderm data is versioned like "git for data"), which can promote collaborative use of Jupyter within an organization on shared and versioned data sets.

Link(s) to additional information:

Release blog post
Jupyter + Pachyderm example

Also, a blog post devoted to Jupyter + Pachyderm is being reviewed and will be released prior to this newsletter.

Links to images or photos to accompany the story:

Jupyter + Pachyderm diagram

Other information about the story:

As mentioned above, a blog post devoted to Jupyter + Pachyderm is being reviewed and will be released prior to this newsletter. I can further clarify any of this information and provide some more links about Pachyderm in general. Just let me know what you think is appropriate, and thank you for considering this story for the newsletter!

Ruv7 commented 7 years ago

Hi @dwhitena - thank you for sending this in. I will include it for consideration in the January newsletter. Feel free to post the blog link once that's available. Hope you're enjoying the holidays!

dwhitena commented 7 years ago

Thanks @Ruv7. I will link the blog post here. It should be published in the next few days.

dwhitena commented 7 years ago

@Ruv7, here is the blog post: https://medium.com/pachyderm-data/jupyter-pachyderm-part-1-exploring-and-understanding-historical-analyses-2a37e56c6578#.4lcljyyzd

dwhitena commented 7 years ago

@Ruv7 Just following up here. Do you have any questions, suggestions, etc. about the above. Looking forward to the opportunity to have this in the newsletter. If it goes out with the newsletter, we will be sure to share on all of our accounts.

Ruv7 commented 7 years ago

Hi @dwhitena - I apologize for the delay in my response - the newsletter was postponed. I'm behind on getting this out the door and I will share an update with you in the next two weeks. Thanks for being patient!

Ruv7 commented 7 years ago

@dwhitena - we're prepping to send out the next newsletter and I wanted to let you know we're including your story above. There has been such a lag in my ability to post the story I wanted to confirm if you'd like to see any changes before it goes. I included the link to the blog post you sent on January 3. Thanks!

Ruv7 commented 7 years ago

@dwhitena - I hope you're ok with me publishing, I expect this to go out this afternoon. I apologize for the short notice :( Though it's not ideal, I can always edit post publishing time so if there is anything you want to see changed just lmk.

dwhitena commented 7 years ago

Yes @Ruv7. That sounds great! Thank you so much.

Ruv7 commented 7 years ago

Awesome, thank you for your patience and for the story idea. I really appreciate you!