search

IHTSDO / rf2-to-rf1-conversion

A utility for converting an RF2 archive into RF1 without reliance on additional information such as a compatibility package
Other
0 stars 1 forks source link

RF2 to RF1 conversion tool for Singapore extension files and to run on bundle that only consists of snapshot #2

Open wankeong76 opened 6 years ago

wankeong76 commented 6 years ago

Hi Peter

We need some help to fix the RF2 to RF1 conversion tool for Singapore extension files and to run on bundle that only consists of snapshot.

In the earlier issue that was closed, you have managed to fix it for US Ext files.

Please help.

Thanks, Wan Keong

pgwilliams commented 6 years ago

Hi Wan Keong. Actually you're the 2nd person to express interest in a snapshot conversion. I've added an undocumented "-s" flag that I've tested with the International Edition to the extent that it will produce an archive, but I've not yet run any validation to see how correct the output is. This is very much "use at your own risk" https://github.com/IHTSDO/rf2-to-rf1-conversion/commit/52ff29542d52b1c487fc7308ff9f7824c85b8574

However, I will need a copy of the Singapore extension in order to make the changes required to work with your files.

Best Wishes, Peter

wankeong76 commented 6 years ago

Hi Peter,

Regarding your point on you need a copy of the Singapore extension in order to make the changes required to work with SG Extension files, can I share the files to you via Google Drive?

If yes, could you provide your Google username/email so that I can share the files out to you.

wankeong76 commented 6 years ago

Hi Peter,

Regarding your point on you need a copy of the Singapore extension in order to make the changes required to work with SG Extension files, can I share the files to you via Google Drive?

If yes, could you provide your Google username/email so that I can share the files out to you.

pgwilliams commented 6 years ago

Yes it's pwi at ihtsdo dot org - hope that makes sense.

wankeong76 commented 6 years ago

Hi Peter,

I have already shared out the Singapore extension files in RF2 to you via Google drive. Please check if you are able to download this zipped file.

This zipped file is password encrypted, I will send you the password separately to your email.

pgwilliams commented 6 years ago

I've done a first pass at adding compatibility for the Singapore extension and have committed it to the develop branch. It runs through and the number of concepts output matches the number in your concept file. The number of descriptions is incorrect, it's just putting through the FSNs which I believe is because it's not realising that the SG language refset file is where to look for preferred terms. The number of Relationship rows is badly wrong - that's where I'll be working next, although I'm not sure when that will be. Any code investigation you're able to do yourselves would greatly help this process.

I ran the code using the following command:

java -jar -Xms3g -Xmx8g target/RF2toRF1Converter.jar -s -v -u /Volumes/ram_disk /Users/Peter/Backup/SnomedCT_InternationalRF2_PRODUCTION_20170731T150000Z.zip /Users/Peter/Backup/SnomedCT_RF2Release_SG1000132_20171115.zip

You can delete _-u /Volumes/ramdisk if you don't have sufficient capacity to create a disk in memory. It does speed up execution if you can set one up. That said, the process completed in 25 seconds, which is amazing compared to doing a full conversion of the International release with history.

wankeong76 commented 6 years ago

Hi Peter,

Thank you for helping us to explore adding the compatibility for the Singapore extension to the converter. We will try to understand more on the converter program logics and see if we are able to investigate on the issues. Could you share out "SnomedCT_InternationalRF2_PRODUCTION_20170731T150000Z.zip" to me via Google drive?

Meanwhile, we will greatly appreciate if you could continue to help us explore solving the descriptions and Relationship issues.

wankeong76 commented 6 years ago

Hi Peter,

May I check if you have managed to resolve the Descriptions and Relationship issues to make the converter work for SG Extension snapshot?