Failed procurement—Now what? // Échec de l’approvisionnement — Que faire maintenant?

[RachelMuston]

The term “failed procurement” describes a procurement process that runs its course, but without a result. This is what happened with the second opportunity posted on the Government of Canada Developers Exchange (GC DevEx). The opportunity closed on April 23, 2018 and did not receive any proposals. Nor did it generate any discussion or comments on the Git Hub issue.

This raises two questions…. Why? And now what?

Was there a problem with the way the opportunity was described or scoped? If so, should Health Canada re-write their opportunity? Was the problem with the skill set required? If so, should we be using promotions to broaden the depth and breadth of the pool of developers who are signed-up to hear about GC DevEx opportunities?

Or is the problem with the GC DevEx platform itself? Do we need to fix any technical issues or processes to make it easier for developers to apply to opportunities?

Please share any thoughts you have in the comments section below!

—

Le terme « échec de l’approvisionnement » décrit un processus d’approvisionnement mené jusqu’au bout, mais sans résultat. C’est ce qui est arrivé dans le cas de la deuxième possibilité affichée sur le Carrefour des développeurs du gouvernement du Canada (CarrefourProgGC). La possibilité a pris fin le 23 avril 2018 et n’a reçu aucune proposition. Et elle n’a pas non plus généré de discussion ou de commentaire sur la question Git Hub.

Cela soulève deux questions… Pourquoi? Et que faire maintenant?

Y avait-il un problème dans la manière dont la possibilité était décrite ou dans sa portée? Dans l’affirmative, Santé Canada devrait-elle réécrire la possibilité? Le problème était-il au niveau de l’ensemble de compétences requis? Le cas échéant, devrions-nous utiliser des promotions pour accroître l’ampleur et la richesse du bassin de développeurs qui se sont abonnés pour être informés des possibilités dans le cadre du CarrefourProgGC?

Ou le problème vient-il de la plateforme du CarrefourProgGC elle-même? Devons-nous corriger des problèmes techniques ou des processus pour faire en sorte qu’il soit plus facile pour les développeurs de présenter leur candidature pour des possibilités?

Veuillez nous faire part de toute réflexion dans la section réservée aux commentaires que vous trouverez ci-dessous!

[RobJohnston]

I thought that it was a bit nebulous... wasn't really sure what they wanted. It didn't even say it it was a Web application or desktop. And what technology stack, other than Event Store?

My suggestion would be to ask for a design first. Once that's in place, then ask for the coding to be done.

But the main reasons I didn't apply (besides a personal time-crunch) was evaluation criteria 4:

Your Experience working with relevant agile projects and work with Event Store...

Could I not just take a Saturday and read the docs?

I guess the agile part explains the nebulous requirements, but it's too hard to judge what the take-home per hour would be when it's all done. I asked myself, "Could I do this in 20 days?". No idea.

[steve-h]

I looked at the opportunity and passed after an hour of investigation. Perhaps I was on the wrong track. Please don't take these blunt views as any personal criticism. At the time I chose to stay silent due to my pessimistic conclusion, since I prefer to work on items with evolutionary value.

The problem was the field of project management. (The ocean) A more specific subset needs to be illustrated with perhaps test data. Smart human project managers often don't solve the problem stated, let alone a $10K computer based solution.

The business stated was for projects of a repeatable type with a waterfall type of execution. Age old potential problem analysis and prevention/abatement should be used. Monitoring for failure seems a little late.

Maybe an EventStore solution would work but I prefer to work from a fundamental "root need" and not from a partially complete solution.

I also used another thought tool: "Pretend the solution requested exists - What would you do next with it?" As an experienced project manager, I could not truly see myself gaining any insight from such a tool. I also could not see it as an objective tool for outside managers. Why doesn't the project manager just flag the failure instead of entering the event and looking at some output before flagging the failure? If I could imagine that the solution worked then I would not do it for $10K as it would be worth millions.

So if you want to advance the problem, start small and with a specific instance of the problem at least.

I had found that EventStore was a niche and realized that I could just learn it easily. My issue was the problem definition was not tangible enough to see the next solution steps.

I only like to work on successful endeavours, so I did not even bid to move a yardsticks in this case. I did not state these opinions when the opportunity was posted, as there is always a possibility that there is a solution and I did not want to taint anyone's innovation. Too bad the "failed procurement" is how it comes to light.

In a "meta" proof, if you had this tool applied to this project, when would it have flagged the failure? And how would it have improved your current failure detection (no bids)? No events was the failure! How does EventStore technology deal with that? :-)

[RachelMuston]

Thanks @RobJohnston. I take your point about the Experience requirement. We recommended some of the wording in the experience section based on wording that the BC Dev Exchange folks used. However their process is quite different from ours. I think we will recommend that GC DevEx opportunities NOT include the 'agile' criteria unless their opportunity is very specifically following an agile model.

I have also wondered whether experience in the specific language/software needs to be a mandatory requirement for every opportunity. Like you say, for some things, reading the docs over a weekend can suffice.

Something to keep in mind for anyone thinking of posting an opportunity!

Thanks for the comments! :-)

[RachelMuston]

Thanks for the thoughtful comments @steve-h. I heard similar things at the ICT Procurement event last Friday... that problem definition is so very important. I agree that test data can be extremely helpful and that making the next steps clear really improves understanding of the problem space.

I like your question of "Pretend the solution requested exists - What would you do next with it?" as a point of reflection. :)

I don't mind that the procurement failure is how these insights are coming to light. In fact I think it is great!!

[karfai]

I left some comments on the original issue (https://github.com/canada-ca/HC-Technology-and-Business-Innovation_Innovations-Technologiques-et-d-Affaires-SC/issues/1) that summarizes what I see are the problems with the offer. All of the comments here are valid, but they don't explore what would actually go in to implementing an application like this one. These comments all assume that the constraints are valid. I don't think they are.

[joel-osc]

@RobJohnston makes a lot of great points. At the end of the day a nebulous project with a $10k budget is a recipe for failure. Think of your $10k budget backwards: subtract time to do all the paperwork to actually get the bid, discover the real requirements, attend meetings, set-up, testing, etc. Now, how much dev time do you really have? 1-2 days likely.

[mgifford]

I don't think that the $10k is necessarily a problem, but it really depends on what is being asked for. The vendor can't be held to necessarily deliver a working product, unless it is a really straight forward request. The vendor can be held responsible for advancing the issue, providing documentation on what they tried and contributing to a broader solution.

But the RFP should probably be no more than 5 pages in length. The response from the vendors should be about the same. The government should consider selecting more than one vendor and being able to compare the delivery of the end result from those selected.

If the challenge is fixing problems with existing solutions that are open, then it is way easier to nudge a big problem ahead with small $10k projects. Using an iterative approach, building in the open, an ecosystem of companies that can support this type of development will happen.

But we have to stop thinking that this is going to happen with these one off procurement opportunities. Doesn't really matter to me if it is a $10k project from GC DevEx or $300k with the Open by Default challenge. This is a small portion of even the innovation investments in Government IT. What would it look like if in 2018 departments pledged to put in $20 million into new procurement and then double that every year till it met just 10% of the total GovIT budget?

Then you'd have real attention from vendors across Canada.

I do think that the expectations of what is possible need to change within government if we are going to successfully pivot GovIT to follow the D5 Charter.

[kelpisland]

@RachelMuston , we’ve had the same challenges with our “code with us” transactions walking the line between being too specific (stifles innovation) and too nebulous (devs don’t know where to start). We (BCDevExchange) decided to refocus “code with us” on the tactical quick hits to deliver code; and introduced “sprint with us” for the more nebulous engagements where a team can be put together to discover and iterate. https://www.bcdevexchange.org If you’d like I can connect you with the procurement side of the BCDevExchange to share thoughts.

[boydjaimie]

@RachelMuston We had a similar outcome, but at a bigger scale, a couple months ago. Why don't you come out on May 30 for our reverse industry day to see if you get some relevant insights from the industry? More info here: https://buyandsell.gc.ca/procurement-data/tender-notice/PW-18-00826453

[karfai]

When I heard about the GC Dev Ex project (via https://www.linkedin.com/pulse/gc-dev-ex-proof-concept-underway-laura-wesley/), I was really excited for this excellent opportunity to collaborate in free software development with my government. Two small projects were posted around that time. Neither had any action from developers. Since then, nothing (publicly visible) has happened.

Has this effort been abandoned or forgotten? Was it simply a means to generate press for a while? I'd be very interested in understanding if this project is still alive.