Wow … almost 3 weeks I didn’t post anything in my blog. Just a quick recap on what I did for the last few weeks. The last two weeks was a bit hectic for me. I need to write up a review of Dreaming in Code for one of my class this term. And to make it more interesting, last week was the final exams week. But it’s done now and time to go back to OpenMRS again.

Last (Couple of) Week(s)

I read through OpenMRS module documentation to get a better understanding on how the module works, especially the web part of a module because we plan to add a few web pages to give user of OpenMRS to use the deduplication module from the web instead of launching a separate desktop application. Right now, I’m done creating some mock web pages (took me a while to understand Spring’s work flow) and we need to hook up the web pages to deduplication process (new feature of the Patient Matching module).

Next Week

The plan for this coming week is adding some new features brought by Shaun from the  conference in Durban, South Africa. The simplest feature that will be added is the capability  to select only one datasource and copy it to the second datasource. This one is not from the conference, I think. But other features that is proposed by Shaun is really interesting. It will give flexibility and power to the user to control the matching process. Attached are the feature that will be added to the Record Matching GUI.

So, enough with the bitching part, let’s do some coding. But, it’s kinda late now. I think I’m going to take some good night sleep and start coding tomorrow morning hehe …

Wow it’s been a while since I post on this blog hehe … Today just a recap of what I did last week. So here it is:

Last Week

What we did in the patient matching project team is mainly discussing about the possibilities of further step that we will take for the second phase of project. Shaun proposed the idea of OpenMRS patients de-duplication module which will beneficial for the OpenMRS (and me because I will have the chance to learn more about the anatomy of the OpenMRS module). Initially there are two options proposed by Shaun Grannis to acomplished the de-duplication process as we can see in the patient matching project wiki page. But in the end, we decided to improve current patient matching module by adding the new de-duplication feature.

So, as the consequences of this decision, I will have to learn about “adding web pages” to the current patient matching module. Originally the patient matching module only work from behind the screen, intercepting method call and wrap it up with AOP. Because de-duplication process will be user trigered process we need to put a page to put “something” that will trigger the de-duplication process.

Next Week

First thing that I must do is reading. Yes, it’s reading time again. This time I need to read the module documentation to find out how to add those webpages. Other devs, Keelhaul, point me out to this extension point page. Both, Keelhaul and bmckown said that it will adding webpages is pretty straightforward (for them hehehe ….). While bwolfe told me to get one of the module from the repos and study it hehehe … Seems to lots of reading documentation and reading codes this week.

PS: bwolfe, bmckown and Keelhaul is the irc nick name of OpenMRS developers. You can find them (and me) at #openmrs inside irc.freenode.net

I had another discussion last Tuesday with Shaun Grannis and James Egg and we think the discussion went really well. This time I didn’t ask too much silly questions hehe … We clarify some more on what we want to do with my first project. There are couple of issues that we focus on, such as how to to propagate the u values from the random sampling result to the EM analysis process.

After doing some digging, I found out that the u value is saved in the MatchingConfigRow object  in the non-agreement property. At the end of the random sampling calculation, this non-agreement  property will be assigned with the result of the calculation. Now we already have the u values from the random sampling process. But how do we propagate this u value to the EM analysis process. Dig some more then …

Well, apparently the EM analysis also take MatchingConfig object as the parameter which contains all above MatchingConfigRow. So, now we need to tell the EM analysis process to use this value when the user want to pick to use random sampling. We need to put a switch then to let know the EM analysis which value to be used, some default value or the values from the random sampling process.

Another thing that we discuss in the phone was connecting this process to the Record Linker GUI. Arghhh, I’m not good at GUI programming. I just don’t have the sense of arts to create a good GUI. But, I have to give it a shot hehe …

Some term explanation:

• Record Linker is the name of the program that I will work on. One of the capability of the program is to combine records from different sources using statistical analysis on those records.
• MatchingConfig is an object that will store the parameter that will be used for analyzing those records. There are lots of parameters that need to be define, for example where to get the records, what fields can be found in the records etc
• MatchingConfigRow is an object that will store the options to match each column in the records. These parameters for example, the algorithm that will be used for the matching process. MatchingConfig object contains series of MatchingConfigRow denoting that a single records will contains many columns in it.
• The random sampling and EM analyzing process will take this MatchingConfig object as their process parameters. This MatchingConfig will be shared by the two process to propagate the result from random sampling to EM analyzing.

Some fact that I learn:

• When the records are coming from file, there are a few step that need to be done before the file can be analyzed. The file are chopped to only include fields that will be used in the analysis process. After the file is chopped, the file is sorted using the operating system built-in sort function on the blocking fields.
• Let’s keep some fact for the upcoming posts hehe …

Any question? I hope I didn’t miss anything …

My first phone discussion about my project with my mentor, Shaun Grannis and James Egg, went well. Shaun and James explain to me about the project in details and I think the project is really interesting. I made a couple of stupid questions that is not related to the project though, sorry for that Shaun and James hehe …

My first project is to implement a fully functional random sample analyzer that calculates the rate of random agreement among corresponding pairs of records between two data sources. This rate value will replace the u rate, field agreement rate among pairs that are truly non-matched, that come from the Expectation Maximization analyzer. To get a better overview about linkage process and rationale behind the process you should read this publication about record linkage. If you want to know more about the Expectation Maximization algorithm you can read the wiki or some other journals and publication.

The process for generating u value for each column are as follows:

• Generate two arrays of Record with the desired size of maximum sampling size
• Take one Record from each array at a time and do the following:
  • For each demographic data in the Record, match their value using selected String matching algorithm (Jaro-Winkler, Levenshtein, Longest Common Substring or Exact Match)
  • If the value from both Record match each other, then increment match rate of current demographic data.
• Do over above process until all record have been paired and examined
• Calculate the u value for each demographic data and set the new u value to the MatchConfig object.

I still need to dig more about the first process and see how each datasource is read and converted into Record object. What do you think about the above process? Did I miss anything?

Woohoo yaaayyyyy … After delayed for a week, Google finally announce the list of accepted students for this year Google Summer of Code and guess what, I’m on the list. Wooohhhhoooo … I was so excited when I saw my name in the list which means I’m accepted as an intern student in OpenMRS for the 2008 Google Summer of Code (GSoC 2008) wooohhoooo yaaayyyy …

Open Medical Record System (OpenMRS) formed in 2004 is an open source medical record system framework for developing countries. For this year GSoC, OpenMRS have so many projects to offer to the students. I’ll be working to Extend Patient Matching Analyzers and Heuristics Support. This blog is dedicated to post any progress that I made on this project.

Right now, I’m still trying to understand current implementation of the patient matching in OpenMRS. My mentor, Shaun Grannis, give me some journals and publications to read in order to understand the matching algorithm. The main idea of the patient matching project is to enable aggregation patient’s data that is scattered in different places with different patient identifier. I think the work will be very interesting and challenging. Burke Mamlin, another mentor in OpenMRS, said that students will have fun working for OpenMRS and I think he’s right. The community is really fun and full of friendly people.