Understanding the Patient Data and Parsing Procedure

[ahmetcihatcetin]

In this issue we'll have a look at the the patient data and how to parse it.

[ahmetcihatcetin]

The Data

The patient data (which is collected from the patients for determining their predisposition to ADHD or the diagnosis of ADHD) will consists of:

• Conners Parent Rating Scale (Numeric)
• Conners Teacher Rating Scale (Numeric)
• [PlaceHolder]

Furthermore, all of the data regardless of their types will be labeled since we will have the diagnosis information of the patients and we are practicing supervised machine learning algorithms in this project.

Let's have a detailed look of these different types of data:

Conners Parent Rating Scale (Numeric)

This questionnaire will be used as the main patient data in the project.

Answers

Conners Parent Questionnaire consists of 48 questions to which patients' parent answers in one of the 4 options:

• Not at all.
• Just a little.
• Pretty much.
• Very much.

Questions

The questions of the questionnaire could be seen below in its entirety:

1. Picks at things (nails, fingers, hair, clothing).
2. Sassy to grown-ups.
3. Problems with making or keeping friends.
4. Excitable, impulsive.
5. Wants to run things.
6. Sucks or chews (thumb, clothing, blankets).
7. Cries easily or often.
8. Carries a chip on his shoulder.
9. Daydreams.
10. Difficulty in learning.
11. Restless in the “squirmy” sense.
12. Fearful (of new situations, new people or places, school).
13. Restless, always up and on the go.
14. Destructive.
15. Tells lies or stories that are not true.
16. Shy.
17. Gets into more trouble than others of the same age.
18. Speaks differently from others of the same age (baby talk, stuttering, hard to understand).
19. Denies mistakes, or blames others.
20. Quarrelsome.
21. Puts and sulks.
22. Steals.
23. Disobedient or obeys but resentfully.
24. Worries more than others (about being alone; illness or death).
25. Fails to finish things.
26. Feelings easily hurt.
27. Bullies others.
28. Unable to stop a repetitive activity.
29. Cruel.
30. Childish or immature (wants help he shouldn’t need, clings, needs constant reassurance).
31. Distractibility or attention span a problem.
32. Headaches
33. Mood changes quickly and drastically.
34. Doesn’t like or doesn’t follow restrictions.
35. Fights constantly.
36. Doesn’t get along well with brothers or sisters.
37. Easily frustrated in efforts.
38. Disturbs other children.
39. Basically an unhappy child.
40. Problems with eating (poor appetite, up between bites).
41. Stomach aches.
42. Problems with sleep (can’t fall asleep, up too early, up at night).
43. Other aches and pains.
44. Vomiting or nausea.
45. Feels cheated in family circle.
46. Boasts and brags.
47. Lets self be pushed around.
48. Bowel problems (frequently loose, irregular habits, constipation).

Digitization

The data will be digitised in order to use(interpret) it in the algorithms of SciKitLearn. The answer options will be digitised as 0,1,2 and 3 respectively:	Not at all.	Just a little.	Pretty much.	Very much.
0	1	2	3

The whole digitised data will be in the form csv. 'Comma-seperated Values' is a data format in which the answers to each question for an individual/observation are simply seperated by commas. We could identify the data as; each individual will correspond to a row meanwhile each question will correspond to column:	Parents	Question #1	Question #2	...	Question #48
Parent of patient #1	0	3	...	1
Parent of patient #2	1	0	...	2

Note that the 'Parents' column is unnecessary and absent in the digitised data we'll use since each row represents a parent's answers. Furthermore, in the digitised data there will be one more column, 'labels' which corresponding to whether or not the patient has diagnosed with ADHD:

Labels
ADHD_positive
ADHD_negative
ADHD_positive
...
ADHD_positive

The Labels column will be crucial for the supervised machine learning algorithm we will use.

The final raw form of the digitised data for Conners Parent Rating Scale (Numeric) of 3 patients could be visualised as follows: 2,0,1,1,2,0,1,0,2,0,2,1,2,0,0,0,0,0,1,0,0,0,2,0,1,2,0,2,0,1,3,0,0,2,0,0,2,0,1,0,0,0,0,0,0,1,0,0,ADHD_positive 1,0,0,1,2,0,1,1,1,1,1,0,1,1,0,1,0,0,0,1,1,0,1,2,1,2,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,1,1,0,2,ADHD_negative 0,1,0,2,3,0,1,0,2,2,0,0,0,0,0,1,0,1,0,0,0,0,1,0,2,1,0,0,0,1,2,0,0,0,0,1,3,0,0,1,0,1,0,0,1,0,0,0,ADHD_positive

[ahmetcihatcetin]

Conners Teacher Rating Scale (Numeric)

This questionnaire will be used as the secondary patient data in the project. Moreover, we are planning to combine the teacher questionnaires with the parent questionnaires into a new data type for the project.

Answers

Conners Teacher Questionnaire consists of 28 questions to which patients' teacher answers in one of the 4 options:

• Not at all.
• Just a little.
• Pretty much.
• Very much.

Questions

The questions of the questionnaire could be seen below in its entirety:

1. Restless in the “squirmy” sense.
2. Makes inappropriate noises when he/she shouldn’t.
3. Demands must be met immediately.
4. Acts “smart” (impudent or sassy).
5. Tempter outbursts and unpredictable behavior.
6. Overly sensitive to criticism.
7. Distractibility or attention span problem.
8. Disturbs other children.
9. Daydreams.
10. Pouts and sulks.
11. Mood changes quickly and drastically.
12. Quarrelsome.
13. Submissive attitude towards authority.
14. Restless, always “up and on the go.”
15. Excitable, impulsive.
16. Excessive demands for teacher’s attention.
17. Appears to be unaccepted by group.
18. Appears to be easily led by other children.
19. No sense of fair play.
20. Appears to lack leadership.
21. Fails to finish things that he starts.
22. Childish and immature.
23. Denies mistakes or blames others.
24. Does not get along well with other children.
25. Uncooperative with classmates.
26. Easily frustrated with efforts.
27. Uncooperative with teacher.
28. Difficulty in learning.

Digitization

The data will be digitised in order to use(interpret) it in the algorithms of SciKitLearn. The answer options will be digitised as 0,1,2 and 3 respectively:	Not at all.	Just a little.	Pretty much.	Very much.
0	1	2	3

The whole digitised data will be again in the form csv. We could identify the data as; each individual will correspond to a row meanwhile each question will correspond to column:	Teachers	Question #1	Question #2	...	Question #28
Teacher of patient #1	0	3	...	1
Teacher of patient #2	1	0	...	2

Note that the 'Teachers' column is unnecessary and absent in the digitised data we'll use since each row represents a teacher's answers. Furthermore, in the digitised data there will be one more column, 'labels' which corresponding to whether or not the patient has diagnosed with ADHD:

Labels
ADHD_positive
ADHD_negative
ADHD_positive
...
ADHD_positive

The Labels column will be crucial for the supervised machine learning algorithm we will use.

The final raw form of the digitised data for Conners Teacher Rating Scale (Numeric) of 3 patients could be visualised as follows: 3,2,2,1,2,1,3,1,0,1,2,1,2,3,2,2,1,2,1,2,1,1,1,1,1,1,1,1,ADHD_positive 2,1,0,2,0,0,1,0,1,0,0,0,2,1,2,1,0,0,0,1,0,0,0,0,0,0,0,0,ADHD_negative 0,0,0,0,0,0,2,0,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,ADHD_positive

[ahmetcihatcetin]

Implementation & Utilization

The csv module for python will be utilized:

• csv.reader(csvfile, dialect='excel', **fmtparams)

  • Return a reader object that will process lines from the given csvfile.

  • Usage in code:
    • Initialize the reader object for csv:
    • csvReader = csv.reader(FileRead1)
• We can iterate over the rows of the csv file by simply doing a for loop on the object:
  • for row in csvReader:
• We could append the fields of a row for a patient onto a list:
  • csvPositivesList.append(row)
    • csvPositivesList is simply a 2D list whose rows contain the fields(columns) of a patient. The list acts as a 'buffer' for positives data: The buffers are used in order to remove patient names and add the classifier field which in this case it is the positive label: ADHD_positive.
    • csvNegativesList: the same description above could be used for this list as well but for the negatives data which are patients data labeled with ADHD_negative.
  • We will replace the last field of a row which is the name of the patient for confidentiality and replace it with the positive classifier:
    • for row in csvPositivesList:
    • row[-1]=adhd_positive
  • The same procedure will be applied onto the negatives data as well.
• csv.writer(csvfile, dialect='excel', **fmtparams)

  • Return a writer object responsible for converting the user’s data into delimited strings on the given file-like object.

  • Usage in code:
    • Initialize the writer object for csv for synthesis file:
    • csvWriter = csv.writer(FileWritten)
    • Write all rows in one call by utilizing csv library:
    • csvWriter.writerows(csvPositivesList)
• At the end of the execution, a new csv file will be created in which the patients' names have been removed and patients' data is correctly labeled. Note that this csv file will contain both positive and negative labeled patients data.

Reference: docs.python.org

[ahmetcihatcetin]

Randomization of Data Order

Since with the execution of the previous parsing code we will get a patient data which has been ordered as positives are first and negatives are second. Thus, by utilizing random module of python, random.shuffle(x) to be precise, we will acquire a ramdomly ordered csv data. Note that for reading and writing the csv files, csv module for python will be again utilized. random.shuffle(x)

Shuffle the sequence x in place.

Reference: docs.python.org