Verbal Presentation Requirements

[mheadd]

In Section 4.08, we need to review the parameters of the verbal presentation from vendors.

[randyhart]

I can't recall what we need to review for this one.

[mheadd]

I think it was just to look at the language and see if anything needed to be modified for this RFP.

[randyhart]

@mheadd @waldoj @sztaylorakgov - we might consider changing is the time limit? Did you all think that an hour was too long for the DPA RFP?

Can you think of any other changes we should make?

[waldoj]

I don't recall thinking that an hour was too long. I mean, it was too long for the vendors who were interviewed needlessly (we all agreed in advance that it was fantastically unlikely that they could say anything that would save them from their bad proposals, but we wanted to interview everybody for the first RFP), but anything more than zero minutes probably would have been too long for them. We ended some in less than an hour, IIRC, because everything was said that needed to be said.

[randyhart]

Good point - I wonder if we should consider some kind of a down-select after reviewing the written props in order to avoid that situation. I'm not sure if this is possible within the state regs, but we definitely knew that a few of the vendors weren't likely to get an award based on their written props.

@susanjabal is this worth discussing or would that complicate things too much?

[susanjabal]

I agree with the concept of pre-screening based on written proposals and only interviewing those respondents that are reasonably susceptible to award. This is allowable by SOA procurement regs; in fact, this is how we normally handle interviews. I am fully open to either way. In the procurement office, we talked about expanding the allowable pages on this one, to see if more useful information is gained in written form - but still interviewing all offerors. Then we could understand how to best move forward on the next round.

[sztaylorakgov]

I'm on the fence about trying to pre-screen. I like the idea of not wasting time, but I also think it will be difficult to develop screening criteria that will objectively/fairly assess and exclude poor responses. I expect @susanjabal that a pre-screening process would be - like everything else - relatively transparent in requirement to respondents. So, whatever criteria we develop, it would just be another hoop to jump in the proposal. Most poor proposals from our first procurement just had a number of bad smells about them. However, there were also some surprises. For example, parts of the proposal from one vendor were very poor, but in the interview and subsequent discussions we've had with that vendor, they've shown a great deal of strength.

Having said that, we only had 6 proposals - which we considered a good number - for the last procurement. If we were dealing with 10, 15 or 20 I would definitely want to pre-screen those. So, maybe we only need to worry about limiting to a small enough number?

[randyhart]

It'd be nice to say "we reserve the right to limit the verbal presentations to the top X vendors". But nailing down the top X takes an extra step for the evaluation team to come to consensus on scoring. I guess Jon G. did have everyone score after Round 1 Written, though? So we could take advantage of the points process that Alaska uses and limit it to a certain number?

Our not including the verbal interviews as it's own eval factor could help us when it comes to doing this.

[sztaylorakgov]

@randyhart That's a good point. Maybe we wouldn't have to develop separate screening criteria. You're saying maybe we just use the scoring as-is pre-interview to rank and include only top X for interviews?

Our not including the verbal interviews as it's own eval factor could help us when it comes to doing this.

You mean we included verbal as its own scoring point value with the DPA procurement and are recommending not giving verbal its own scoring point value for this RFP, yes? I remember discussing this a bit a while back. The only concern I have with that approach - and it's probably minor - is that if something comes up in the verbal that is important, but doesn't relate to the other scoring elements, then we don't have a good way to impact score. Not sure this is really a thing. So, not sure we want to address it.

[randyhart]

Yes - I could have worded that much clearer! Your takeaway is what I was trying to say!

[susanjabal]

I agree with all of the above, and we can certainly say that we intend to host verbal presentations only with vendors reasonably susceptible to award. In real-life, what this would look like is: we would evaluate/score the written proposals utilizing the criteria identified in the RFP (no need to develop more criteria). the resultant scoring would determine who is 'in the running' and we would host presentations with those. If an offeror has no chance of being awarded, even if they gain the max points from the verbal presentation, they are not included. If we decide to proceed with this plan for this one, I would set aside some points for the verbal presentation, as we had done in the DPA RFP. As I mentioned above, this is how we normally handle this, so appropriate language is already templated.

[ghost]

I hate to lengthen this discussion, but just for completeness there is one alternative method of screening that hasn't been mentioned. It is basically a kind of veto, where a vendor passes on to the next step unless there is obvious and overwhelming evidence that they're just not going to make it. This kind of thing is done in hiring interview processes all the time. The assumption is that a candidate will go through the entire process unless it becomes obvious for some reason that they are not suitable and the rest of the screening process would simply be a waste of time. So there's that for consideration.

[susanjabal]

Touching base on this one - Alan, I think your suggestion of a veto is very much in line with what is being proposed above - with the exception that SOA utilizes a point system for scoring. The points would automatically enact the 'veto' once it is clear that the vendor has no ability to achieve award due to their scores. Does that make sense? Or I am I mis-interpreting your suggestion?

I will incorporate language stating that we may not interview everyone (we could always choose to still interview everyone if we deem appropriate; this will just give us options) if all agree.

[randyhart]

I agree.

[susanjabal]

I am closing this one, as language allowing us flexibility to interview some or all respondents will be incorporated.