API development with date filter and access to private zone data

[bezunakarmi]

PPA etenders system developers, DAAC System Integrators to develop the required API that will facilitate automated data transfer to the Open Contracting website.

[mireille-raad]

any update on this process? Is it gonna part of the existing automated API update and data assessment report?

[mireille-raad]

almost 2 months later... i am happy to say that DAAC are working on this. There are 2 main tasks they are doing:

• publishing the additional fields.
• Changing the endpoints to have data for one month and statistics about the month (how much data per stage is there)

so... this should take care of a huge burden of having to import gigantic amounts of data... I will let you know once they finish and I test... this should be the last tweak we do.

[mireille-raad]

We just heard back from DAAC and they are done with this tasks :D

What they have done is:

• publish additional fields from the private zone
• create a monthly end-point (so data can be imported by month) - each end point has statistics about it. (detailed below)

So for our to do list:

• can we adjust the import to work from the monthly json? the python script need to check for what month we are in and just import the data for the month instead of the many many years. This can solve our memory and server crashes problem. Each month will have a statistics json, we can use the stats to double check that all import was done successfully.

• Activate the placeholders in the database. As per previous conversations, the placeholders are ready in the database and we need to add the fields to the view.

• Let me know if you have questions or clarifications i can ask DAAC.

@anjesh i've emailed you a word document with the same information - feel free to forward it to the team.

Updated API details

Statistics.

URL: http://etender.gov.md/json/YYYY/MM/statistics Example result : { year: "2017", month: "5", intentionAnounceList: 74, tenderList: 383, contractList: 905 }

Planning stage

URL: http://etender.gov.md/json/YYYY/MM/budgetList http://etender.gov.md/json/2017/5/budgetList - valid example http://etender.gov.md/json/2017/05/budgetList - valid example This format covers all entity requests.

Additional fields

No.	Field	Result
1	Budget / Source	budgetItem.source (at this time will be null until change main functionality)
2	Budget / Published on	budgetItem.publishDate
3	Budget / Bulletin	budgetItem.bulletinNumber (bulletinPublishDate)

Tender

URL: http://etender.gov.md/json/YYYY/MM/tenderList

No.	Field	Result
1	Tender / Value	tendetrItem.tenderData.expectedBuyingSumm
2	Tender / Award Criteria	tendetrItem.awardCriteria
3	Tender / Award Criteria Name	tendetrItem.awardCriteriaName
4	Tender / Eligibility Criteria	Missing in eProcurement
5	Tender / Tenderers / Identifiers	tendetrItem.tenderers.code
6	Tender / Tenderers / Identifier / Id	tendetrItem.tenderers.id
7	Tender / Tenderers / Identifier / Legal Name	tendetrItem.tenderers.fullName
8	Tender / Tenderers / Additional Identifier / Id	Missing in eProcurement
9	Tender / Tenderers / Additional Identifier / Legal Name	Missing in eProcurement
10	Tender / Tenderers / Address / Street Address	tendetrItem.tenderers.cAddress
11	Tender / Tenderers / Contact Point / Name	tendetrItem.tenderers.director
12	Tender / Tenderers / Contact Point / Email	tendetrItem.tenderers.email
13	Tender / Tenderers / Contact Point / Telephone	tendetrItem.tenderers.phone
14	Tender / Tenderers / Contact Point / Fax Number	tendetrItem.tenderers.fax
15	Tender / Procuring Entity / Contact Point / Name	Missing in eProcurement
16	Tender / Documents / Id	tendetrItem.documents.id
17	Tender / Documents / Title	tendetrItem.documents.docName
18	Tender / Documents / url	tendetrItem.documents.url
19	Tender / Documents / date published	tendetrItem.documents.currentStatusDate
20	Tender / Documents / format	tendetrItem.documents.fileName

Awards

Several offers belonging to different bidders can be win. With each of which a separate contract is signed. Therefore, both the Award and the Contract are one and the same element of the list of contracts. URL: http://etender.gov.md/json/YYYY/MM/contractList

mapping

No.	Field	Result
1	Award / Date	contractItem.contractDate
2	Award / Suppliers / Identifiers / Id	contractItem.participant.id
3	Award / Suppliers / Identifiers / Legal Name	contractItem.participant.fullName
4	Award / Suppliers / Address / Street Address	contractItem.participant.address
5	Award / Suppliers / Contact Point / Name	contractItem.participant.director
6	Award / Suppliers / Contact Point / Email	contractItem.participant.email
7	Award / Suppliers / Contact Point / Telephone	contractItem.participant.phone
8	Award / Items / Quantity	contractItem.winningPositions[wPosIndex].quantityAwarded
9	Award / Items / Unit	contractItem.winningPositions[wPosIndex].offerPosition.unitMeasure.*Value (I.e. mdValue or ruValue or enValue . For example unitMeasure.mdValue)
10	Award / Documents / Id	Missing in eProcurement
11	Award / Documents / Title	Missing in eProcurement
12	Award / Documents / url	Missing in eProcurement
13	Award / Documents / Published Date	Missing in eProcurement

Contract.

URL: http://etender.gov.md/json/YYYY/MM/contractList mapping

No.	Field	Result
1	Contract / Period / Start Date	contractItem.contractDate
2	Contract / Items / Id	contractItem.winningPositions[wPosIndex].offerPosition.id
3	Contract / Items / Description	contractItem.winningPositions[wPosIndex].offerPosition.goodsName
4	Contract / Items / Classification / Scheme	contractItem.winningPositions[wPosIndex].offerPosition.goods.classificationScheme
5	Contract / Items / Classification / Id	contractItem.winningPositions[wPosIndex].offerPosition.goods.code
6	Contract / Items / Classification / Description	contractItem.winningPositions[wPosIndex].offerPosition.goods.*Value (I.e. mdValue or ruValue or enValue . For example goods.mdValue)
7	Contract / Items / quantity	contractItem.winningPositions[wPosIndex].quantityAwarded
8	Contract / Items / unit	contractItem.winningPositions[wPosIndex].offerPosition.unitMeasure.*Value (I.e. mdValue or ruValue or enValue . For example unitMeasure.mdValue)
9	Contract / Amendment / Dates	contractItem.amendments[amendIndex].finalDate (This is the new end date for the contract period (contractItem.finalDate) if the contractItem.amendments[amendIndex].contractType = 4” Extension of validity period” or 6”Other”)
10	Contract / Amendment / Changes	Missing in eProcurement
11	Contract / Amendment / Type	Missing in eProcurement
12	Contract / Amendment / Amount	contractItem.amendments[amendIndex].amount (The original contract amount (including all previous changes) will be increased / decreased by this amount if the contractItem.amendments[amendIndex].contractType = 1” Price increase”(amount > 0) or 2”Decreasing price” (amount < 0) )

[timilsinabishal]

@dc-coder Thanks for the detailed mapping. However we have some confusions, we cannot see the tenderItems field in tender object. The closest match to tenderItems is tenderData but it is object instead of Array. Going through previous data, we can find there are multiple items in a tender. Is tenderItems missing in the api or is there anything we are missing ?

[mireille-raad]

I tried to explain the issue and showed the text of the tickets... it was a little bit hard to translate.

Can you re-write this in a more simple english... along with an example or screenshot of the json in question? what the problem is, what it should be?

I will try to communicate this to DAAC as easily as possible

[bezunakarmi]

The issue that we face is that we couldn't find the items/goods array in the API. When using the tender API http://etender.gov.md/json/2015/01/tenderList, we found a tenderData object which contains a single good object.

According to our previous API http://etender.gov.md/json/positionList?id=25631, we used to get JSON object with array of goods.

So, are we suppose to expect an array of goods or is it single good associated with a tender?

[timilsinabishal]

@dc-coder We are using new monthly JSON API for updating http://moldova-demo.yipl.com.np. This has made updating process much faster, but there seem to be minor differences in data between monthly API and old API (eg. there are fewer contracts, tenders in monthly API). Also, the monthly API is showing error recently.

[mireille-raad]

hey! thank you!

sorry for taking 19 days to respond to you :) - i was on summer vacation without internet connectivity.

My question is:

• are you now able to find items/goods array in the new API? is this problem resolved or still the same?
• can you give me an example of less data in the monthly API?

I will email with DAAC about it. Thanks a ton!

[bezunakarmi]

API is working well. But still there is only one item/goods per tender. Therefore, we are using the old API for tender items.

We compared the totals using both old as well as new API, we found a slightly fewer data in the monthly(new) API.

It would be helpful if you could contact DAAC about these issues.

[bezunakarmi]

@dc-coder any update on this issue?

[mireille-raad]

sorry it took a while to respond... i am on another work trip and it has been a bit crazy.

I will try schedule a call with DAAC to have discussion in person. I am afraid doing this troubleshooting with DAAC over email won't be effective. IF a call happened, would you be willing to join in?

Sorry again and thanks a ton :) i seriously deeply really appreciate it. :bowing_woman:

[bezunakarmi]

Hope everything is fine at your work now. It will be great we can resolve this issue soon. Let me know when you are planning to call DAAC, I'll try to schedule my calendar accordingly.

Thanks.