Editing the Notebook: R4ML_Data_Preprocessing_and_Dimension_Reduction.ipynb

[MadisonJMyers]

Alok, it is very clear that you are skilled and knowledgeable about what you are writing about! I want to help you better communicate that to the reader. I can see a lot of thought went into the layout and organization and I must compliment you on that! There are some grammatical and formatting errors that I suggest you implement, however. Please see my comments below:

• Preparation and Preprocessing are redundant. Please select one for your title.
• I believe the journey team wants number formatting to have a period after (1. , 2. , 3. etc.) Please change the numbering to be in accordance with that.
• Change the first two sentences in the introduction to “In our first notebook we received an introduction to R4ML and conducted some exploratory data analysis with it.”
• Change the second and third lines to: “In this section, we will go over the typical operations a data scientist performs while cleaning and pre-processing the input data. Now, we will look into dimensionality reduction, first starting out with the boiler plate code from Part I for the purposes of loading etc.”
• Put a colon after the fourth line.
• Combine those lines into a paragraph.
• In 2.1 remove “to do”
• Under 2.1, bullet point one remove the comma and value should be plural (values)
• Lower case or capitalize the beginning of each bullet point
• End each bullet point with periods for consistency
• “model” should be “model’s”
• Remove the comma from the fifth bullet point
• Remove section 2.2. It is unnecessary.
• Under 2.3 “User” should be “The user’
• In the third bullet point of 2.3 “it’s” should be “Its” and you should remove the “more”
• All sentences should end with punctuation.
• All sentences should start with a capital letter.
• Lowercase the “And” in the second sentence of 2.4
• Under 2.5 remove “As we can see that,” and capitalize the “all”
• Remove the space between the word and the comma (“ (explained later), the”
• “Himself” should be “independently”. We are not speaking to solely males.
• Remove “Here we will list a few of comming techniques” and “Other data pre-processing steps, that are typically done”
• A period is needed to end the sentence under 2.5 before “Alternatively”
• A space is needed after “lambda, “ and the “This” in the parenthesis should be lowercased
• A period should end the sentence before “For the practical…”
• This entire paragraph is confusing. I would try to rephrase it and then I can look at it again afterwards. Try reading it out loud to see if it makes sense.
• In your box “Outlier Detection” the D should be capitalized
• In the same box in the fourth row “ne can use use the” should be changed to “Use the”
• The next column has “effect: misspelled. Please correct.
• In 2.6 “spend” should be “spent”
• “data scientist” should be plural (data scientists)
• Is Dimensionality Reduction a completely separate step?
• In 3.1 remove “typically”
• “These allow you” should be “All of these allow you”. There should also be a space before that sentence.
• Remove “These are the benefits”
• Remove “Visualization of lower dimension data and it's interpretation are more straightforward and it coule be used for geting insights into the data.” It seems out of place. Perhaps rephrase it if you disagree.
• Remove “with an example” from the 3.3 title
• “for big data set” should be “for a big dataset”
• Put a colon after illustration and change “For instance” to “For example” (“For example, see the following illustration:)
• Under PCA transformation after “variance:”, lowercase the “The”
• The portion “Since components describe specific directions in the data space, each component depends by certain amounts on each of the original variables: Each component is a linear combination of all original variables.” Is confusing. Try to rephrase or delete.
• Under “An Example” change “dependency” to “dependencies”
• Change the “?” to “.”
• Remove the second sentence as it is unnecessary. (“Another question is that if we can reduce the dimensionality.”)
• Change “As we can analytically, see that we need first 6 principal component.” To “Analytically we can see that we need the first six principal components.”
• Change “Lets” to “Let’s”
• Change the “one” throughout this notebook to “you”
• In the last sentence under 4. “In the future notebooks” should be “In future notebooks” or “In the future notebook” depending on if it is one or more.

[aloknsingh]

fix as per review

[xwu0226]

@aloknsingh It would be ideal if @MadisonJMyers has a chance to do another check over your changes for accuracy and close the issue herself. Thanks!

[MadisonJMyers]

After review, I've found a few minor edits:

• Under 2.2 I don't see the diagram you had shown me about time of the data scientist. Will you be adding onto this section?
• 2.3 "How R4ML address data preparation and pre-processing?" should be: "How does R4ML address data preparation and pre-processing?"
• "The function is called r4ml.ml.preprocess and it supports various feature transformation routine e.g R4ML supports all the major pre-processing task" should have "routine" be plural and "task" should be plural
• In 2.4 capitalize the beginning your sentences and end them with punctuation.
• "However, we want to end this topic with the note to the user that using the custom DML (explained later), the advanced user can write the advanced data pre-processing step independently." should be slightly changed to: "However, we want to end this topic with a note to the user that in using the custom DML (explained later), the advanced user can write the advanced data pre-processing step independently."
• "Other data pre-processing steps, that are typically done." should end in a colon not a period.
• "Box and Cox proposes a family of transformations that are indexed by a parameter lambda, (this feature will be available in future) For the practical one can calculate the kurtosis and skewness for various lambda. or one can do the significance testing to check which lambda gives the better result." needs edited punctuation. (period after "future)" and remove the period before "or"). Also I would suggest changing "For the practical one can" to "Practically, one can...".
• In 3.2 "is problematic because" should end in a colon.
• Capitalize your sentences that start with a bullet point.
• Under "PCA transformation", "variables: Each component" the "Each" should be lowercase.
• "How many eigenvectors we need?" should be "How many eigenvectors do we need?"
• Under that section, "We will try to find it out in the following code" needs a colon at the end.

Other than that the notebook has improved a lot and flows well! Great work.

[aloknsingh]

Hi Madison

in ref to "Under 2.2 I don't see the diagram you had shown me about time of the data scientist. Will you be adding onto this section?"

it's there I am not sure why you can't see it

[aloknsingh]

fixed as per comments

[MadisonJMyers]

Hm I'm not sure why the diagram isn't showing? Can others see it? If so perhaps it is my browser or something else. Other than that I only noticed one edit:

• In 2.3 and 2.4 capitalize the beginning your sentences and end them with punctuation.

Once that's complete I will close the issue! Great work!

[xwu0226]

I don't see it either. Is this the place?

[MadisonJMyers]

Yes that's where it should be. Alok showed me in person, but I'm not sure why it's not showing up on our computers. I'm using Firefox- what browser are you using?

[xwu0226]

neither Chrome nor Safari shows

[MadisonJMyers]

Hm, have we encountered this problem before? I'm not sure how to fix it. @aloknsingh any ideas?

[xwu0226]

maybe Alok showed it in DSX ? and the diagram can be only rendered well in DSX jupyter? directly showing the notebook file in browser may have some issue? @aloknsingh

[aloknsingh]

may be you want to double click?

[aloknsingh]

@MadisonJMyers can you pl have all the comments at one shot rather than incremental thanks

[aloknsingh]

this is direct link https://dataplatform.ibm.com/analytics/notebooks/fa911700-b14a-47d8-a8e9-4b8f82a0d0da/view?projectid=9f9a9e62-8ff0-42ef-a285-4bdff61e18cd&context=analytics

[aloknsingh]

done

[MadisonJMyers]

Yeah, I'm giving you all of my comments together at one time. My comments that follow my original comment, are in response to your comment "fixed as per comments". When I see that you have claimed implementation, I then review to make sure nothing was missed. When I reviewed, I noticed a few errors, which I posted.

Regarding the diagram: you cannot "double click" it. There's no button. When trying to access your DSX notebook, the access is forbidden.

2.3 and 2.4 are still missing periods.

[aloknsingh]

Ok.

on the review side. there are many instance where we use period in end and not period in end. Why this differences (wrt to initial comments)? I think we should either have all period or no period in the header and sub-header. Also if we see general paper or book usually there are no period in heading. But they do include ?

[MadisonJMyers]

Great question!

So all sentences that are not in the header (which is what I was referring to in 2.3 and 2.4) need to end in punctuation. Usually it's a period but in instances when you are going to follow up with a list, you need a colon.

I agree that consistency is important! I usually ask for a period to end bullet points, but not headers. Others may ask you to do something differently. I think, like you pointed out, consistency is most important.

All that's left on your end for this is to end your sentences in 2.3 and 2.4 with periods. (There are two sentences without them.) Once that is complete, I'll close the issue.

If any of this is ever confusing, don't hesitate to reach out to me and ask questions. I'm happy to help!

[aloknsingh]

I was referring to other sections like in the

1) Introduction and Exploratory Data Analysis -) there is no period in header "Introduction" and it's sub section -) similar there is no period is other places

2) Data Preparation and Dimen -) There is no period in Introduction

3) Classification Model using SVM -) There is no period in any of the 1 2 3 sections

I think in general if you open any textbook they don't put period in the header or sub-header and it is considered bad practice to put. so in that spirit I suggest lets put no period in the header , subheader

[aloknsingh]

@xwu0226 and @MadisonJMyers seems like browser can't render all the element of the ipynb. It works fine in DSX . here is a preview .download these files

https://github.com/SparkTC/r4ml/pull/55/files

[MadisonJMyers]

@aloknsingh if you disagree, feel free to get someone else to review in addition to me. To be clear, I am not asking you to put periods in your headers. I am simply asking you to close all of your sentences with punctuation. I will close the issue once this is complete.

[aloknsingh]

Ok, I see your point. i think I will modify 2.3 2.4 and then change all the header to be non period. I think this should be consistent. How does it sounds

[MadisonJMyers]

Great! Let me know once you've done that.

[MadisonJMyers]

I see that you've implemented the two periods. Awesome. I'll go ahead and close this issue, but I'd recommend having it reviewed once again as some phrasings are still awkward. I also recommend figuring out why your diagram in 2.2 isn't showing up for people. If this issue persists, you may want to remove that section. Other than that, good work.