synthetichealth / synthea

Synthetic Patient Population Simulator
https://synthetichealth.github.io/synthea
Apache License 2.0
2.16k stars 645 forks source link

Some Issue probably with my CSV file? #690

Closed kumarmb closed 4 years ago

kumarmb commented 4 years ago

Hi

I am trying to create Medrecords for India. As suggested, I created the following files corresponding to India. • demographics.csv • timezones.csv • zipcodes.csv • hospitals.csv • primary_care_facilities.csv • urgent_care_facilities.csv files.zip

I have attached the demographics file here. The state and city I use in the command below are present in demographics, zipcodes, hospitals files. But I am getting the error

java.lang.Exception: The city Roha was not found in the demographics file. at org.mitre.synthea.world.geography.Location.(Location.java:60) at org.mitre.synthea.engine.Generator.init(Generator.java:188) at org.mitre.synthea.engine.Generator.(Generator.java:143) at App.main(App.java:144)

Below is the command I am using. I edited the Demographics file using Excel and saved it as CSV. Not sure whether that is causing the error

Few other notes:

The Demographics file has data for only one state Gujarat (about 5000 records) whereas the zipcodes file as well as other files have 150K plus records corresponding to all states. I had to do this to avoid heap size error I am getting..

./run_synthea Gujarat Roha

Any help will be greatly appreciated

Thanks Bharath

kumarmb commented 4 years ago

And to add, if I do only /run_synthea Gujarat then I get this error

Population: 1 Seed: 1585520434988 Provider Seed:1585520434988 Location: Gujarat Min Age: 0 Max Age: 140 java.lang.NullPointerException at org.mitre.synthea.world.geography.Location.assignPoint(Location.java:281) at org.mitre.synthea.modules.LifecycleModule.birth(LifecycleModule.java:179) at org.mitre.synthea.engine.Generator.generatePerson(Generator.java:350) at org.mitre.synthea.engine.Generator.lambda$run$2(Generator.java:275) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) {alive=0, dead=0}

jawalonoski commented 4 years ago

@kumarmb If you have a debugger, try putting in a breakpoint at org.mitre.synthea.world.geography.Location.assignPoint(Location.java:281) to see what is happening. Or up above that in LifecycleModule.birth.

My guess is that you have a disconnect between one of the files. But it is hard to test without all of the files, because I'll just end up getting other exceptions without the provider files.

Have you looked at the synthea-international repository for working examples of how to do this? Also, we'd love for you to contribute the India files there when you are ready!

kumarmb commented 4 years ago

Thanks, I figured it out at least partly. It appears that more of the files under Providers need to be changed - Primary care, urgent care and hospitals. I created those and I am able to run but one city at a time but not for the whole state. I am working on it and I will also look at the international link as well.

./run_synthea -s 21 -p 10 Gujarat Patan works but ./run_synthea -s 21 -p 10 Gujarat does not. I am debugging it now

Yes, I will upload the India files. The main demographics and zipcode files are created from actual Census data but the hospitals data is simulated data

kumarmb commented 4 years ago

Hi Jason

Thanks for your help. I got it to work - as expected the issue was with the data. What I am doing now is to split the file demographics.csv into smaller files and then run. Otherwise it gives Java heap space error. I am not a Java expert but I couldnt run with the whole file, may be the settings need to change

I have uploaded the two files into Git

Thanks Bharath

On Mon, Mar 30, 2020 at 5:58 AM Jason Walonoski notifications@github.com wrote:

@kumarmb https://github.com/kumarmb If you have a debugger, try putting in a breakpoint at org.mitre.synthea.world.geography.Location.assignPoint(Location.java:281) to see what is happening. Or up above that in LifecycleModule.birth.

My guess is that you have a disconnect between one of the files. But it is hard to test without all of the files, because I'll just end up getting other exceptions without the provider files.

Have you looked at the synthea-international https://github.com/synthetichealth/synthea-international repository for working examples of how to do this? Also, we'd love for you to contribute the India files there when you are ready!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetichealth/synthea/issues/690#issuecomment-605982974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5NZ7E7B6N4LJKBY5PPGC3RKCJOBANCNFSM4LWEE7FQ .

kumarmb commented 4 years ago

Spoke too soon. Where should I upload the files to? Under synthea international, it says uploads are disabled... I am somewhat new to github as well, so may be i am doing something wrong

On Sat, Apr 4, 2020 at 12:27 PM Bharath Kumar kumarmb@gmail.com wrote:

Hi Jason

Thanks for your help. I got it to work - as expected the issue was with the data. What I am doing now is to split the file demographics.csv into smaller files and then run. Otherwise it gives Java heap space error. I am not a Java expert but I couldnt run with the whole file, may be the settings need to change

I have uploaded the two files into Git

Thanks Bharath

On Mon, Mar 30, 2020 at 5:58 AM Jason Walonoski notifications@github.com wrote:

@kumarmb https://github.com/kumarmb If you have a debugger, try putting in a breakpoint at org.mitre.synthea.world.geography.Location.assignPoint(Location.java:281) to see what is happening. Or up above that in LifecycleModule.birth.

My guess is that you have a disconnect between one of the files. But it is hard to test without all of the files, because I'll just end up getting other exceptions without the provider files.

Have you looked at the synthea-international https://github.com/synthetichealth/synthea-international repository for working examples of how to do this? Also, we'd love for you to contribute the India files there when you are ready!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetichealth/synthea/issues/690#issuecomment-605982974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5NZ7E7B6N4LJKBY5PPGC3RKCJOBANCNFSM4LWEE7FQ .

jawalonoski commented 4 years ago

@kumarmb You can't push the files directly, first you have to "fork" the synthea-international repository, and then issue a pull request. See https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork