Open source Pocket server-side code

[davidar]

I was redirected here by https://bugzilla.mozilla.org/show_bug.cgi?id=1343006

Are there any plans for open sourcing the server-side code of Pocket?

I realise this isn't really the right repo to be filing an issue against, but there doesn't seem to be a more appropriate repo available (yet).

[mkoidin]

@davidar Thanks for checking in.

We continue to work on open sourcing more of Pocket, but we don't have specific plans around the server-side code to share at this time.

[satishkn]

Hi, Without the server-side code, the application would not be functional. Correct? Also, can you share links to the architecture/documentation of pocket ? Thanks

[mcshanti]

I continue to be deeply curious about the backend code behind the pocket codebase. The outstanding tickets about open sourcing the pocket code - in addition to the original news release, are almost two years old. Is there a rough timeline for open-sourcing the server code?

[mkoidin]

@mcshanti Thanks for checking in.

We are planning to open source some server side code later this quarter related to some code that supports the Firefox browser. More details to follow when we are ready. We don't have anything else to share at this time related to a broader server side code release.

[wizzwizz4]

@mkoidin Could you link to the announcement, please? I can't find it.

[worldofgeese]

@mcshanti Thanks for checking in.

We are planning to open source some server side code later this quarter related to some code that supports the Firefox browser. More details to follow when we are ready. We don't have anything else to share at this time related to a broader server side code release.

What happened to this promise? It's been 3 years since the original announcement was made to open-source Pocket. Please do right by your users.

[br4nnigan]

we don't have specific plans around the server-side code to share at this time.

then I don't have specific plans on using your extention. Actually I'm not using Firefox any longer.

[penyuan]

Thank you @mkoidin for being the only Pocket person to have responded so far, but I will echo @worldofgeese that it has been three years since Mozilla promised to make Pocket fully open source and it still hasn't happened.

By keeping Pocket closed source, proprietary software (and this includes the whole stack from server, client, to architecture, documentation and specs), Mozilla is behaving in a way directly antithetical to the 10 Principles in the Mozilla Manifesto. In particular, Principles 6 and 7.

Here's hoping that we will get a more direct and concrete response from Mozilla with regards to why they are keeping Pocket proprietary.

[mkoidin]

@worldofgeese Here is the code I referred to in the earlier post you referenced: https://github.com/Pocket/proxy-server

@penyuan I am Pocket's General Manager (and previously CTO). I've been with Pocket since its founding nearly 10 years ago. So you definitely have the attention of the right person and I appreciate the frustration.

To provide a bit more context. Pocket and our code existed before Mozilla and was not open source. Now that we are a part of Mozilla we’ve been taking steps to open source. We’ve made some progress, but completely appreciate that the speed is not what some would like to see.

We’re still a small team and we do not have the bandwidth to spend full time effort on open sourcing our historical server code, but as we’ve developed new services we’ve been working on open sourcing. With the code we are open sourcing it’s more complicated than just adding a license and making the Github repo public. We need to conduct security audits, legal/IP audits, we need to review 10 years of code history, etc.

Last fall, we open sourced the Pocket proxy server code that we use to deliver sponsored content to Firefox New Tab.

We had plans coming into 2020 to do a bit more, but our progress this year has been slowed due to COVID-19 and the need to adjust our priorities.

That is the update for now, we will continue to release at github.com/Pocket as we go.

[worldofgeese]

@mkoidin I want to start by thanking you for speaking plainly and in good faith. I love Pocket, love Mozilla, it's for this reason I so passionately hold Mozilla to the standard of its own manifesto.

I know Mozilla seeks a diversified revenue stream and that Pocket is one of its few assets not tied to Google's purse strings. Certain people in the community suggest Mozilla might be delaying the complete source release in a shrewd move to protect its financial interests. Can you speak to this hearsay?

I want to see Mozilla succeed. People like me are the ambassadors putting Firefox on our family's machines, teaching good password management practices (Lockwise), and the importance of keeping web browsing private (Firefox Private Network). I hope Mozilla continues to be an example of leadership in ethical software for many decades to come.

[mkoidin]

@worldofgeese I can say unequivocally that this has nothing to do with "protecting financial interests" — it has never been a part of the conversation. Pocket's ability to generate revenue for Mozilla is completely decoupled from whether we are open source.

As I mentioned above it has always been a matter of bandwidth and how our team balances the move to open source with other responsibilities to our users, Mozilla, etc.

[helmut72]

I guess some people ask for server code to run a self-hosted private system. Releasing this or that peace of server code only is useless.

[shantiii]

@mkoidin Thank you for responding and giving the transparency that you have thus far. I can sympathize with working iwth limited bandwidth, especially given just how many extenuating factors there have been in the time since the original comments. I'm coming at this as a fellow developer who very much "gets it", but also hopes to run a private instance of this software for myself (and possibly my friends and family).

As it is right now, do you think you could give an updated estimate of when/how long open sourcing the server code would take? If not, is there a way us folks could effectively influence the priority of that work?

Additionally, part of the strength of Mozilla is its ardent and often skilled fan community. I don't labor under delusions of person-months, but would you be able to take on volunteer talent operating under an NDA to assist in the open sourcing effort? I'm sure you'd find no shortage of volunteers in this thread, self included.

[mkoidin]

@helmut72 re: self-hosted. Completely understand. I believe there are some open source alternatives out there they allow for self hosting, but I can't speak to any of them directly.

@shantiii Appreciate the offer! And certainly understand that part of us getting more of this out there is that it will make it easier for us to enable the community to engage directly on the code base. Unfortunately, outside of the code we've already released at github.com/Pocket, I don't have a way to support adding volunteer/NDA talent to assist. Interesting thought though and will keep in mind.

[penyuan]

Just a quick response to add to the thanks from @worldofgeese. I think many of us here share @worldofgeese's sentiment in that I "love Mozilla, it's for this reason I so passionately hold Mozilla to the standard of its own manifesto."

On that note, can @mkoidin and Mozilla at least re-affirm their promise - in accordance with the Mozilla Manifesto - to release the whole Pocket stack as open source even if a specific ship date is TBD (though hopefully ASAP)? Thanks! :smiley:

[andyw8]

For those interested in building an open source Pocket, please check out https://github.com/open-pocket/open-pocket

[Oppen]

I appreciate open sourcing may be a multi-year project (it has been until now at least) for the reasons @mkoidin listed. Would it be too much to ask if instead of having to ask periodically maybe a semi-regular update be posted somewhere for those interested? Specially since I see the repos are active, but being microservices I cannot tell how much of the service is already open. For all I know everything is open now but I could find no announcements about it.

[wwijkander]

I found https://github.com/Pocket/pocket-monorepo which is ~4 months old while looking into this, FYI.