PardeeCenterDU / IFs-Issues-Tracking

This repository only holds the list of bugs that have been reported for IFs. Anyone may add a bug report, but please look to see if your issue has already been added!
2 stars 0 forks source link

Education forecasts for Guinea Bissau (IFs Base) #71

Open wverhagen opened 3 years ago

wverhagen commented 3 years ago

The education forecasts for Guinea Bissau seem to be on the optimistic side, especially for changes in education quality. there are a couple of related issues: 1) the initialization is not from data. Yutang and data team are currently looking into unesco data for Guinea Bissau 2) the forecasts are rather optimistic and seem to pertain even after economic growth is artifically reduced

the issue was found in IFs vb6 version 7.70, but also seems to be present in earlier versions

mtirfan66 commented 3 years ago

Thanks for this, Willem.

"the initialization is not from data. Yutang" - I am not sure why it is assigned to me though. The proper assignment would be to Yutang or data team. Correct me if I am wrong on this, Willem.

"the forecasts are rather optimistic." Please give reasons why it seems optimistic. For example, take other countries which start with about the same quality score and similar levels of development as measured by GDP per capita and/or years of education. See, how these countries unfold over time, whether Guinea Bissau is moving faster than such peers.

I have identified one issue with the Guinea Bissau students costs for secondary. The pre-processor brings consistency between the per student costs of total secondary, lower secondary and upper secondary. The secondary cost (total) variable was reading a very old data point which was unusually large. This was shooting up the initial cost in lower and upper sec. As these costs were being brought down to the stablization point from the regression function, there would be more money and more students in secondary. I have fixed that initialization. This lowered the ed years and quality forecasts for Guinea Bissau. I am still working on it one other issue. I see an spending/allocation mismatch in education for a few countries, e.g., Guinea Bissau, Somalia. This does not look right. I see other countries (Bangladesh) where the two matches. I am working on it.

The three figures below, identify countries that start at a similar point of quality score for adults (total) and show quality and years (the main driver) for them between old and changed base with the code changes I mentioned above. As you can see, Somalia is still a problem, while Guinea Bissau is better. I will need to switch to poverty age-sex work later today and might come back to education only later in the week.

image

image

image

image

image

mtirfan66 commented 3 years ago

I see now you also assigned Yutang, Willem. I was not sure who quciet is :)

wverhagen commented 3 years ago

Thanks Mohammod for looking into this.

For the 20 years development has lacked significantly behind general development trends in sub-Saharan Africa. Across a host of reports, the general expectation is that that will continue into the near to not so near future. Except if major policy changes are implemented. I've looked at ed quality relative to SSA. See below, for both primary, secondary, and overall adult test scores Guinea Bissau does not only make up its distance to its peers in africa-west and sub-saharan africa but quickly surpasses it. Maybe that was already fixed with your more recent updates

image

image

image

On education years this looks somewhat, although again guinea bissau is making up ground relative to SSA, but that has not been a trend for the past 20 years.

image

Another question is on the relation between 15+ education years and 15-24 education years. One would expect main growth to come from new 15+ adults that have higher education than the historical forecast. Now what could be driving education years if the 15-24 group is lower than the average, and how can it be that the initialization is lower than the average. This is clearly not the case for africa-west or sub-saharan africa.

image

We are seeing a faster rise in education years in ssa compared to other regions, but I'm not seeing that in for example the lower and upper secondary completion rates. These look reasonable to me, but they don't seem to be driving a convergence towards either SSA or africa-west. So why is that convergence coming in education years?

image

Because we are seeing it in adult completion where there is a convergence of guinea bissau to the other groups.

image

Another point of attention, and maybe this is just me not understanding the education model is in primary education from 2020 to 2050: net enrollment in primary in guinea bissau < SSA gross enrollment in primary in guinea bissau < SSA over time primary completions rates in guinea bissau > SSA

I don't understand how lower enrollment can result in higher completion rates

I think the main idea was for this week to have an assessment of the size of the issue. Your assessment was their might be not be one. Mine is that there might be one. Maybe we can talk about that. The next step would then be "how much time does it cost to update this". We still wanna have this done by next week

Hope that helps, happy to talk more about it

mtirfan66 commented 3 years ago

My results are different than yours already, Willem, because of the model changes I mentioned in my earlier comment. I found a second issue on the financing side and decided to look at it before sending code for consolidation.

On the comparison between SSA and Guinea Bissau, group forecast is a good first step in validating results. I follow any anomaly found in the comparison between a country and its peer group by a deeper exploration at the country level. I have done that country-comparison for the quality forecast for working-age adults you identified earlier. I have started looking at the new ones you sent. As you might already know the group aggregates might be somewhat confusing specially with series with skimpy data. For example, for your first one, only South Africa, Botswana and Ghana has data for 2005, GB does not. Comparing the forecast for these countries with Ghana, it doesn't look like Ghana is growing faster even in comparison to its income-level peer, Ghana. It looks like they are initialized a little higher though. I will check into the initialization.

Your timeline is seeing tight now. But I'll try my best. Let's talk about that and the issues in Skype.

image

image

image

mtirfan66 commented 3 years ago

I pasted the third plot to show how South Africa is lagging behind Ghana as their edyears stagnate. Will doublecheck.

mtirfan66 commented 3 years ago

On the initialization of quality scores - we are using regression functions driven by educational years. The functions are good fit relative to many others we use in IFs. There are two problems with the data - a. countries that take the tests are usually high education countries, b. quality scores seem high for the low education countries that take these international tests, apparently because the tests are taken by the more advanced students in these countries rather than random samples.

image

image

image