Feedback on Pay for Pull

[bmann]

I am opening this issue as a central place to discuss / give feedback on the Pay for Pull initiative http://blog.data.gov.bc.ca/2016/01/pay-for-pull/

[sintaxi]

Great idea. I'd be curious to know how it will be dealt with when a PR is only partly functioning/complete as is the case with most PRs.

[bmann]

@sintaxi don't assume anything. What I've heard right now is that "the best" code will be awarded.

So that is a problem, and is actually counter to how open source works: it will lead to duplication of effort and multiple people posting solutions, rather than working together.

I also don't know how we should use this thread. Technically, something like "This is spec work" should be an issue thread, and another could be what you asked "How to deal with a partial PR". @sintaxi can you do me a favour and file a separate issue for that question?

[sintaxi]

yep. done. #27

[Stv]

Primary feedback: The way this is currently written sounds like this is spec work - 99 designs for code. There's other models that could plausibly both provide some protection for people working on it, while still allowing for individuals/small orgs to bid/win on BC gov projects. They reference the 18F experiment (https://18f.gsa.gov/2015/11/06/micro-purchase-lessons/), which has some of it's own problems ($1 bids).

[StormTide]

Initial feedback:

To some extent I'm excited about this concept, but I'm also worried that its not sufficiently removed from the normal expectations of an rfp and a payday model to mirror the innovation pace expected from the open source approach. It might be talking like open source but acting like government procurement.

1. Does the $1000 bounty have enough scope to truly try out this as a purchasing mechanism -- in the context of 100 million investment fund in the VC startup model, and in the context of the openStudent model development scope, I'm curious how one generates enough pay-for-pull opportunities to actually give this model a fair trial. Eg does it scale or launch with sufficient scope and critical mass to even allow for success or comparison?
2. At small values there is created an expectation on the public service side that the issue is already fully described and that there is a clear understanding of what a solution looks like. Is this aligned with bidirectional and collaborative projects? How can the community create bounties for work they would like done to publicly-sponsored projects? Most open source projects that are sponsored by an organization end up having leadership roles given to non-organizational members. How does this work in the BC Gov project model? How does this tap into the innovation ability of the private sector? As an alternative, when Netflix went to community innovation model they put forward a million dollar prize http://www.netflixprize.com but accepted only one solution among a field of competitors.
3. A way to say 'ive got this one' is going to be key. Keeping it from being gamed is going to be harder. Derby-style programming is probably not going to be a successful model.
4. Collaboration mechanisms are needed -- certain PR's are certainly going to involve multiple disciplines -- but if the model is going to try to treat GitHub as a resume, then split credit or co-authorship will be important.
5. PayPal seems like an odd way to pay someone. Lets add the standard payment options, direct deposit, p-card and so forth. I know the govt knows how to use those techs because they do it whenever they want taxes collected or refunded or just want a snack from Mac's http://www.proactivedisclosure.ca/?q=macs+conv ;)

[bmann]

@StormTide great feedback. I would suggest we keep this main thread as loose discussion, and have people create separate issues for any item they themselves "care" enough about to keep pushing forward.

Note: I'm trying to help out and have suggested using GH issues, but am not directly involved with the project, just trying to be a GH herder.

[StormTide]

I think I'll let them get the project launched before opening any issues.. but its a good idea to have a feedback thread.

[carchrae]

+1 on this sounding like 99 designs for code. unless these jobs are very quick and/or there is no competition, the EV is terrible to consider spending time on it if the motivation is payment. but maybe there are stackoverflow types who will do it for glory and beer money, or lovers and dreamers who do it to make the world a better place.

and if this is open source work; will the issues that get PRs be useful to other people? write a custom script to import some horrible government data format, clean file, bla bla - that kind of work is more elance than open source. and i don't think i've ever read a government tender that wasn't something like that - innovation is explicitly shunned in pretty much every RFP i've had the joy to read.

i applaud the govt for trying to innovate but i would like them to think about how i (or anyone) could support a family on what they are offering. yes, i know this is an experiment, but the scaling equivalent of this seems to be akin to gambling for a living.

...

aside from fame and glory, the main business reason to support open source is that the company has already paid for the work (aka no/little extra cost) and you will benefit from others reviewing/testing/contributing to it. there is also the general idea that you should give back, but cynical me will seldom use that when trying to convince business guy on that basis.

[carchrae]

also, it seems a bit strange to post an issue like this: https://github.com/bcgov/citizen-engagement-web-toolkit/issues/7

if it is about extending an existing open source toolkit, why not post the issue in that project? or if that doesn't work why not (a) fork it, (b) post issue (c) accept PR (d) submit PR to the original project.

posting in the original project would be far better as then you would attract the relevant expertise.

[neilk]

I added a comment on the blog, but restating some of it here:

+1 on @StormTide's remarks. There's the extra weird problem that if the project is public, I think the PR has to be public. So developers could see a PR, improve on the approach, and resubmit for a shot at the $1000. If there's a deadline, competitors would probably wait till the last second, to avoid that.

I don't see a requirement that code be licensed under an approved Open Source Institute (OSI) license. The LICENSE for the linked sample project just says "multiple licenses" (which is fucked up, it leads to completely unusable code repos, but that's another story).

Also, will developers have to be registered somehow as government contractors? I assume not?

[dbhume]

I’m David—one of the leads for the Developers’ Exchange. This feedback is fantastic and we hope to hear more. Thanks for joining us in the experiment.

A couple of things we wanted to make clear:

1) Our aim is to make it easier to do business with government. We want to see successful developers as well as successful government programs, and the concerns we’ve heard about spec work are ones we’re taking seriously. We want to build respect and opportunity for developers, not undermine them. 2) This design is by no means done. We’re aiming to improve it, so keep the feedback coming. 3) That said, as we keep going and working through different approaches, I don’t know that we’ll ever land anywhere that satisfies everyone. But let’s keep pushing through to get the best result possible.

We are definitely watching this thread. If we’re slow to respond, it’s because you’ve raised a question we need to think about or pin down details, for example @neilk’s comments on licensing. So we’ll be back at you as soon as we can.

I’d like to move this great conversation to our bcdevexchange/rapid-adoption repo so it is linked to place where we are building the pay for pull wiki. Any objections?

[bmann]

@dbhume no objections on moving, just that you have many many many repos and you might want to think about having a main discussion & docs repo and have a process for directing feedback there. That being said, no easy answer to that, since you want discussion close to where code is being done, too.

[deaves]

Okay, so I couldn't see a discussion over where Hume asked us to post and didn't want to be post there out of context. I'll cross post if someone tells me where to go.

Some thoughts: 1) @bmann I don't think you are right about Open Source eliminating duplication of work. Quite the opposite. I've seen many instances where people work on problems in parallel. Indeed, for people using Open source to teach, one of the biggest challenges, is that someone fixes the problem before your student can. So I don't know if I totally get that. What is critical is that when a solution is posted and adopted it becomes the new brick upon which future work is done. 2) The nut of @StormTide's (valid) concerns seems to be - is this going to be a marketplace - where people compete to create code, or is it going to be a community, where people choose tasks and seek out help. I've seen open source communities behave as either. 3) There are tons of questions about this initiative and if it will work. The creators acknowledge this by stating over and over that "this is an experiment." They probably should have used the word beta.

My own sense is: a) the culture of this experiment is going to matter a ton b) the people - the alphageeks - participating in this thread have a potential to have a HUGE impact on that culture c) that culture needs to be super supportive of experiments and trials, so if the BCDevExchange people get it wrong or create an outcomes that we don't like (like say turning this into Mechanical Turk versus a community) we need to talk about it in supportive ways as opposed to screaming about it, and discourage those who do scream d) get more data by encouraging bcdevexchange to post work and talk about what happened so we can test which hypothesis people have above are real, versus the ones that are imaginary.

[deaves]

Oh, and quick follow up. -1 on the PayPal concern. This is not a hypothesis that needs testing right now. BCgov people should wait to see if anyone actually submits code, if it is good enough and how to get that working. That is the hypothesis that matters (and what I hope this thread focuses on). If the BC gov people have 100 "resource points" they should all be going to getting the incentives and culture right. Spending time building out another way to pay people when we don't even know if they want to do this yet seems anti-lean. We should debate payment systems once we have multiple users that need to paid and the core hypothesis has been deemed proven,

[StormTide]

@deaves paypal thing would count me right out off the batt. Ebay (PayPal) still has a $1000 bug bounty sitting payable to me because I wont go to the headache of doing a corporate resolution to open a PayPal business account. Might get a few sole proprietors, but its not like PayPal is a standard account for your average small coding business, and its non-trivial to setup.

[howardroark]

So I'm just a passer by... Are these issues about this site?

Is there an open endeavour anywhere that aims to shape the site itself, or these sort of efforts in general? Or does that happen behind closed doors? I watched the video... I guess I sorta got it's intentions. Compared to the caliber of products rolling out these days, it really doesn't hold up. To be honest it feels like the byproduct of bureaucracy.

Where did the feedback come from that these issues are about in the first place?

(I've worked for many agencies that have had gov/banks clients and the biggest enemy of a good product has always been the political/hierarchal abyss)

[adelevie]

Hi! I'm one of the devs at 18F working on the micro-purchase platform.

I love that British Columbia is trying this out, too! I hope we can learn from each others' experiences. If you do decide to go the auction route, our platform is open source (standard Rails app) and we would be thrilled if BC gave it a spin.

[kelpisland]

This issue was moved to BCDevExchange/rapid-adoption#2