PardeeCenterDU / IFs-Issues-Tracking

This repository only holds the list of bugs that have been reported for IFs. Anyone may add a bug report, but please look to see if your issue has already been added!

2 stars 0 forks source link

Data Importing & Vetting features in IFs #245

Closed quciet closed 8 months ago

quciet commented 10 months ago

Some more issues we discovered during the data vetting process,

When we do comparisons between newly imported tables and existing tables in IFs, some existing tables are not properly loaded. You can see that in the screenshot below.
When we are doing a comparison between a newly imported table and an existing table (Mark Differences between New and Old Data), the following error will happen,
A few times (I say a few times because it only happens twice for me and now I'm not able to replicate the issue), an error happens when we try to plot a country between old and new data,

If you want to run some tests, you can take the file attached here, IFsDataImport.zip

PardeeCenterIFs commented 10 months ago

I couldn't reproduce #1, it looks fine in my version, could it be the WDI update that I assume you have in your version?:

PardeeCenterIFs commented 10 months ago

On #2, I was able to reproduce it and fixed it, it'll be included in version 8.05, but I don't see anything marked, could you help me identify something that should be mark after running this option.

PardeeCenterIFs commented 10 months ago

3, I couldn't reproduce either, and it seems doesn't always happen for you, these are the types we need to record on video, in case it's a long path that takes you to the error:

quciet commented 10 months ago

On #2, I was able to reproduce it and fixed it, it'll be included in version 8.05, but I don't see anything marked, could you help me identify something that should be mark after running this option.

Thank you so much! I'm not sure actually. The purpose of this function is to find a difference in values of same country-year between old and existing tables. I do believe Sudan or South Sudan, or Kiribati should have big changes.

quciet commented 10 months ago

I couldn't reproduce #1, it looks fine in my version, could it be the WDI update that I assume you have in your version?:

Maybe it is just my version. I will try a new version later next time.

solox1 commented 10 months ago

Hi Yutang,

I see there's a couple of updates in the Vetting form from 8.03 to 8.04, maybe that's what fixed it, but I don't think it was the same issue.

On Tue, Aug 29, 2023 at 3:56 PM Yutang @.***> wrote:

I couldn't reproduce #1 https://github.com/PardeeCenterDU/IFs-Issues-Tracking/issues/1, it looks fine in my version, could it be the WDI update that I assume you have in your version?:

[image: image] https://user-images.githubusercontent.com/33665510/263824367-6c6bc04c-8769-4e32-aa3c-5e1e4c4cde92.png

It was reported by one of the RAs using another data update. I used this WDI update and saw the same problem. Is it possible that this is some bug being fixed from 8.03 to 8.04? I'm using 8.03

— Reply to this email directly, view it on GitHub https://github.com/PardeeCenterDU/IFs-Issues-Tracking/issues/245#issuecomment-1698197958, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIZP4RKPYERP6PE2SKQUOKDXXZQPJANCNFSM6AAAAAA4BWIZMI . You are receiving this because you were assigned.Message ID: @.***>

PardeeCenterIFs commented 10 months ago

On #2, I was able to reproduce it and fixed it, it'll be included in version 8.05, but I don't see anything marked, could you help me identify something that should be mark after running this option.

Thank you so much! I'm not sure actually. The purpose of this function is to find a difference in values of same country-year between old and existing tables. I do believe Sudan or South Sudan, or Kiribati should have big changes.

Maybe I had taken this update before, because I don't see any differences in Sudan or South Sudan, at least for XDebt%GDP.

quciet commented 8 months ago

Hi there,

Another potential issue was found when we were importing the data, IFs is filling in values for country-years that were originally blank (examples below). This issue occurs during importing (example below). The data is already blended so it’s not IFs blending data, it looks like IFs is interpolating/extrapolating data from one year to subsequent years that should be blank. Also, the most recent column was picking up values, but wasn’t always accurate. I attached the spreadsheet below used for data import, and you can see the resultant table would have some filled in values. For example, values of Albania in 1990-1994 were added by IFs.

RoadsPaved%_OriginalBlended.xlsx

Any idea? Thanks!

PardeeCenterIFs commented 8 months ago

Hi Yutang, wow, I don't know how I didn't see this one before, it's basically taking the previous value instead of the nulls. I have fixed it and the fix will be included in version 8.09.

I'm going to close this issue, but you can open a new one if there's something pending from this long thread.