Closed hawc2 closed 2 years ago
This is a solid tutorial @thomjur. It’s very useful for exploring and teaching a specific method for data analysis. There are places I identify below where clarifications and reorganizations will help guide the reader. The key revisions to make before I send this out for peer review are:
Comments organized by Section follow below. Can we aim to receive a revised draft back from you by January 31st? Please note Programming Historian editors will be taking a two-week break starting Monday, December 21st until January 4th.
I think any comments I would make about the latter half of the tutorial are mostly going to reinforce what I’ve said above. Overall the code works, the analysis makes sense, you deal with all the technical issues fluently. At times, you could afford to provide more brief statements defining terms, summarizing ideas. If you get to the code sooner in your tutorial, some of the opening sections explaining K-Means Clustering would be easier to follow as you cite the specific example in your code to explain the ideas behind the analysis.
The last two sections, “Standardizing the data” and “Summary” both seem too long without enough signposts guiding the reader. Add more subsections – for instance, you could include a separate section on “How could we proceed from here?” where you could also discuss some of the alternative approaches you mentioned earlier in the tutorial. These last sections are really the most important part of the tutorial, so guiding the reader through these steps carefully, explicating ideas and the code in more detail, is what will be most important.
Dear Alex @hawc2,
Thanks again for your constructive comments. I agree with all of them (ok, one tiny exception, see elbow-method).
Let me start with the four major issues you mentioned (plus some general comments):
Responses to section comments:
Thanks again for your help, and I am looking forward to your comments!
Best, Thomas
Added a revised version of the tutorial (minor changes, nothing content-related). Exchanged two images (ax labels were cut off in the former version).
Thanks @thomjur. We're sending this out for peer review. You'll hopefully receive feedback in the next month or two.
Alex @hawc2, thanks for the opportunity to review this tutorial. @thomjur, this is a very useful and accessible tutorial about using k-means clustering in Python. I believe someone who is not a humanities scholar and wants to learn k-means clustering will also have interests reading this tutorial. This tutorial will contribute to the goals of Programming Historian. The draft is in good shape and I have the following comments.
Major issues:
Minor issues:
Hi @thomjur, I want to begin by highlighting things that are working well in this tutorial. Your explanation for how to run k-means clustering with scikit-learn is mostly sound and clear — well-done. I also like that you use and discuss variable scaling — that's great! Lastly, I appreciate that you are not using random or artificial data but a dataset that has an actual connection to the humanities.
However, I believe that there are some significant issues with this tutorial (as it is currently written), and they mostly stem from the choice of dataset.
Big points:
[x] While I appreciate your attempt to find a relevant DH dataset, the Der Neue Pauly authors data does not seem well-suited for k-means clustering. The data does not contain coherent clusters from a mathematical perspective and from an intuitive perspective. Additionally, the features — word count and article count — are not particularly interesting or informative. I could imagine an improved version of this tutorial that instead considered the vocabulary of each article in the DNP (words converted to numbers via TF-IDF).
[x] Relatedly, the research question driving the tutorial and motivating the use of k-means clustering — to measure scholars' impact in a specific domain — is not as strong or compelling as it could be. I think it's important to anchor this tutorial with a stronger question that needs k-means clustering to be answered.
[x] Based on the above two points, I would recommend choosing a different dataset and different research question for this tutorial. Believe me, I know it's hard to find a great example dataset, but I think if you do, it will be a huge service to the DH community and ensure that DH researchers and teachers are using k-means clustering appropriately. A lot of the potential alternative datasets that I keep coming back to in my mind are related to text documents that have been converted to numbers via TF-IDF. I'm not sure if you want to go that route, but that could be one possibility, and if you still wanted to plot the clusters, you could reduce the data with PCA or t-SNE.
Smaller points:
[x] The tutorial dedicates a lot of space to parsing the differences between supervised vs unsupervised vs reinforced machine learning. I think a sentence or two would suffice there, and you could spend more time discussing the k-means algorithm in particular.
[x] The tutorial links to Wikipedia for a visualization of how the k-means algorithm works, but I think a visualization within the tutorial itself would be super helpful (perhaps you could even include that GIF or a similar one with credit to the original authors?).
[x] The tutorial instructs the reader to set the random_state
parameter to 42 but does not explain the significance of random state or why it should be set to 42 (a common convention in machine learning supposedly inspired by Hitchhiker's Guide to the Galaxy). I would suggest that you either forgo the random_state
argument entirely, or, if you choose to set a random state, that you explain the concept and why you are choosing 42.
[x] This is a tiny point, but the tutorial explains that k-means work well for "unknown data," but I feel like digital humanists often know things about their data but want to uncover unknown structures in order to gain a new perspective on their data. Perhaps you could spend a sentence or two explaining the significance of k-means for this audience.
Thanks to @hluling and @melaniewalsh for your thorough reviews of this lesson. @thomjur, I'll leave it to you to address the minor revisions, as they seem straightforward. As for major revisions, @hluling's one suggestion seems easily doable. @melaniewalsh's revision advice will require some change to the lesson's structure and length, but I think you can do it without rewriting the lesson entirely.
It is ok if you find it too difficult to change the dataset for this lesson, but it is still important you explain why you chose this dataset in particular, and how K-Means clustering works and doesn't work for this dataset. More importantly, Melanie's suggestion that you use TF-IDF to enhance your analysis seems worth doing.
Along the way, you can more clearly clarify how this process can be used to answer research questions with K-means clustering of textual features. In the end, the most important thing isn't that this lesson perfectly answers its mock research question, so long as it explains how researchers could use it to answer various types of research questions with varying amounts of success.
Your lesson is currently only approximately 4000 words, so you have a lot of room to expand as necessary. I'm optimistic that some additions to the lesson to address the reviewers concerns will result in a lesson that will prove more persuasive to readers trying to decide when to use K-Means clustering over other algorithmic methods for answering their research questions.
In particular, I think the conclusion could be expanded to better address the issues reviewers raise, giving more time to explain what is useful and not useful about K-Means clustering. Throughout the lesson, you could also weave in more comparisons of K-means with other methods - this will help better draw out for the reader why this method is appropriate. Like Melanie says, it seems less necessary to discuss in detail the differences between supervised and unsupervised machine learning, and more time differentiating K-Means from other algorithms like PCA that offer comparable analyses.
Please try to make your revisions in the next month, or two at most. Let us know if you'll need more time, and please feel free to add here any further thoughts you have as you proceed with revisions.
@hluling @melaniewalsh @hawc2
Thank you all for your constructive suggestions. I am happy to revise my tutorial. However, I am still struggling with the major issue raised by @melaniewalsh. Even though I like the idea of using TF-IDF and actual documents for K-Means clustering, this would mean changing the entire article because such an analysis (including TF-IDF) is simply not possible with the current dataset (which was trying to mock customer data analysis, a viral example to illustrate clustering algorithms).
Thus, as far as I can see, I need to change the dataset in order to fully cope with all your suggestions (since, as I said, implementing TF-IDF is simply not possible with the current dataset). I am willing to do this, but I will first need to think of a dataset that is neither too complex for a tutorial (which would be the case with thousands of documents with very different sizes from Brill's New Pauly) nor poses any legal issues (which would most likely be the case with scraping all articles from Brill's New Pauly as well; it's a protected website with paid content). Of course, it should also make sense actually to analyze the data. I will ventilate this question for a while and see if I can come up with something. I have some first ideas, but I am not sure whether they will actually work. Of course, I will then need to rewrite the tutorial (that is based on the current dataset) and change the code, which is fine. However, I am not sure if I can do this within a month or two (family, covid, and a non-existing vaccination strategy in Germany are a bad combination). However, I honestly like the idea of using documents with K-Means clustering, so I see what I can do (maybe I'll have some sort of inspiration). Ooohhhhmmmm....
I will keep you updated, and thanks again for your evaluation!
Thanks @thomjur for clarifying. I'd be curious to hear what @melaniewalsh or @hluling would say. Please chime in if you have any recommendations or further feedback.
For my part, I think it's possible you could work with this dataset and find another way to make your analysis and results more robust. You do a good job in this tutorial of discussing how to pick a number of clusters, but the transitions between method explication and dataset seem to quick, leaving vague why K-Means clustering is right for this particular dataset and research questions. I do wonder if there's another approach with this current dataset that might work as an implementary or subsequent step.
Your section, "How to Proceed?", in particular, opens more questions than it answers, and is one place I'd encourage you to expand the lesson. You write, "Usually, it is a good idea to implement different clustering algorithms, such as hierarchical clustering and k-means, and to compare their results. Even though we have skipped this part in this tutorial, it should be pretty easy for you to add other algorithms now that you have understood how to work with scikit-learn." To me, this is something this tutorial should go into in some detail, to show why K-Means is appropriate here. Perhaps that is an alternative to a new dataset/TF-IDF route, if that proves too difficult to rewrite into the lesson.
Regardless, considering this will require major revisions, it is fine if you need more than two months to get the final draft in shape. Keep us updated on your progress, and have a good spring!
@thomjur @hawc2 @hluling
I hope it's ok that I discussed this tutorial in some detail with my research group, but they also feel that the tutorial is solid except for the choice of dataset. I still think that incorporating TF-IDF could be useful and cool, but I think what's most important is that whatever dataset you choose has real clusters in it (e.g., in the plot, there would be space between the clusters). One of the drawbacks of K-means clustering is that it will identify clusters even if your data does not contain clusters, as discussed at the beginning of this StackOverflow answer. And, from my perspective, it's important that readers know what kind of data is appropriate to use with this technique.
I'm sorry that this suggestion requires more revision work in such a difficult year, and I can empathize with wanting to get this piece published more quickly because there's so much good material here, but that's my take on the tutorial. @thomjur if you choose to go with another dataset, I would be happy to brainstorm or discuss alternatives. I'm excited about this tutorial and think it could be really useful for the community!
Thanks a lot for your encouraging response, @melaniewalsh!
No worries, I am happy about every kind of feedback that I can get, so I am glad that you discussed my draft in your group. And I am also excited to revise my tutorial. The more I think about it, the more I have to agree with what you suggest. My "lazy" defense, so far, was that real-world data oftentimes does not include strictly separated clusters. But even though this might be true, it does not make sense to use such data in a tutorial. Still, I think that it is, generally speaking, reasonable to use K-Means with such data, and if it's to demonstrate that your customer/author data does not include any clusters (and I agree, two to five outliers are not really a "cluster"), which could also be a valuable insight.
I will think some more about textual data that could be interesting to analyze with K-Means. Due to legal aspects, I tend towards Wikipedia; yet, Wikipedia is also somewhat overused, although maybe not to the same extent as Twitter data.
I am also struggling with this whole data/privacy thing... I am not sure if this is due to my German angst background (since Germans tend to exaggerate this whole data privacy/protection stuff, IMHO), but I see potential problems/lawsuits everywhere.
Glad to hear we're coming to a consensus.
@thomjur, as far as copyright/privacy goes, feel free to email me some of your thoughts. I can guide you through deciding about those copyright/privacy issues, and I'll consult with other Programming Historian editors if necessary. Wikipedia is certainly safe, and not infrequently used by PH, in part because it is more likely to ensure sustainable lessons. The ideal dataset would be one familiar to a large audience, and appropriate to produce meaningful clusters with K-means.
@thomjur @hawc2 @melaniewalsh Sorry to jump in late. I'm glad to see the consensus too. I just wanted to point out that clustering techniques are exploratory in nature. I believe this tutorial reflects a common process whose goal is to explore any possible patterns without knowing whether there are 'real' clusters or not before running the analysis. I agree with the StackOverflow post mentioned by @melaniewalsh that we should examine the required assumptions before running an analysis. However, when there are four or more variables so that it is difficult to visualize the raw data, checking all the assumptions for K-means is not feasible (maybe there are formal tests of these assumptions that I don't know). So my position is "we won't know until we try something out." The current dataset and the exploratory research question, as shown in the current version of the tutorial, do not preclude the use of K-means clustering. That being said, I do agree with @melaniewalsh @hawc2 that the ambiguous result may be problematic for publication due to a lack of coherent narrative. So I'm excited to see a revised version of the tutorial no matter which suggested route is taken.
Just a quick update from my site: I hope to send you a revised version of my tutorial by the end of this week.
@hawc2 @melaniewalsh @hluling
I have just uploaded a revised version of my tutorial. Since I have changed major parts of the tutorial, I will not respond to every minor point mentioned by the reviewers.
Major/Minor changes:
I am looking forward to your comments!
There seems to be problem displaying the tables. Is there a way to fix that, @hawc2? If not, I could simply delete the longer ones.
@hawc2 @melaniewalsh @thomjur I’m glad to see the added text-based dataset and a much more expanded discussion on different techniques involved in clustering. I like the added visualization in the section that explains how k-means clustering works and why standardization is needed, as well as the added parts on the silhouette scores. I also appreciate the detailed explanation about DBSCAN.
I think this version meets publication standards as a tutorial. I just have the following comments for improving clarity.
Major issues:
[ ] On feature selection (para. 68 and para. 70), I don’t think how this is done is clearly explained for a reader. I understand the goal is to reduce the number of features from ten to three based on the silhouette score, but I don’t get how this is done exactly based on my reading. Maybe adding a two-to-three-sentence summary of progressiveFeatureSelection( ) would be helpful.
Specifically on progressiveFeatureSelection( ), why did you set the default n_clusters as 3?
[ ] Similarly, the section of finding the optimal eps value may be unclear to a reader. It took me some time to figure the whole thing out by reading the towardsdatascience piece and the piece cited in the towardsdatascience piece. Regarding para. 56, after “we can plot the distance between each data point in a dataset and its nearest neighbor,” maybe just add “we then sort the distance in ascending order.” I suppose this would make it easier to understand Figure 11. Also, for Figure 11, please add the axis titles.
[ ] I think the big takeaway from DBSCAN is that the result does not lead to groundbreaking findings. This is fine for a tutorial. But I’d like to see a more concrete and forward-looking suggestion for the next step of analysis. For example, what do you think would be some new research questions (para. 118) generated for the next step of research?
Minor issues:
@thomjur @hawc2
I think this revision improves the tutorial a lot. The Greco-Roman author clusters are more coherent, the use of PCA is nice and very helpful, and there seems to be a better communication of how clustering works with real data vs artificial data (e.g., "As we can see, there is no real elbow in our plot this time"). The author has clearly has put a lot of work into this revision. Well-done!
Similar to @hluling, I like the addition of text data, TF-IDF, and DBSCAN. However, it also makes the tutorial lengthier and more complicated. I hesitate to suggest this because I know it would be even more work, but it occurred to me that this version of the tutorial could usefully be split into two parts: Part 1 (K-means) and Part 2 (DBSCAN). That could be one way of capitalizing on all this great material while also paring things down for two easier, more focused reads with two different datasets and clustering methods. Would that be possible?
My comments will mostly focus on the K-means section, but the general concepts apply to the DBSCAN section as well.
Major issues
For instance, there might be groups of authors who are discussed at length but to whom limited manuscripts are attributed. On the contrary, other groups may include authors to whom many surviving manuscripts are ascribed but who only have short entries in the DNP. Another scenario could be that we may find groups of authors with many early editions but only a few modern ones.
Maybe the possible research questions could come after the sentence "The original German version of the DNP has been translated into English (starting in 2002)." It might also be helpful to mention/discuss specific authors more, to ground the research questions even further.
Minor issues
[x] Figures 2 and 3 use red centroids, while Figure 6 uses yellow centroids. It might be good to keep the centroid colors consistent. It might also be useful to put a little cross over the centroids to further visually distinguish them (you could just manually add that in Photoshop or Preview, etc.)
[x] Paragraph 2 and 3 could be condensed, perhaps concluding with the sentence "The original German version of the DNP has been translated into English (starting in 2002)."
[x] In the Jupyter notebook, you might mention that yellowbrick
needs to be installed, since it doesn't come with popular Python distributions like Anaconda, or you might even include a pip/conda install yellowbrick
line
[x] I agree with @hluling that author name labels would be very useful for the figures!
Thank you for your constructive criticism, @hluling @melaniewalsh @hawc2.
I agree with most of your comments, and I will try to implement them as soon as possible. Most of them are pretty straightforward (adding axis labels, adding a couple of sentences here and there, etc.).
However, particularly Melanie mentioned a couple of things that I would like to discuss first.
I'd be happy to add some additional sentences or even tables to the analysis. Yet, I am somewhat hesitant to dive too deep into the discussion of these still mock-up use cases. This is simply not a research article, and although I find the results sort of interesting too, I am not sure if this text isn't already too long for a tutorial (depending on if people reading it are looking for a concise recipe of how to use clustering or want to know more about Greco-Roman authors).
Regarding the second point mentione dby Melanie: I personally thought that it might be a good idea to explain the concepts based on a reduced and somewhat adopted dataset. This allows showing the general idea behind these methods more clearly. Also, including only two to three dimensions (features) make the visualization a lot easier. This section is meant to show "how it is supposed to be/look like" (in an ideal world with perfect data) and the actual analysis "what it usually looks like." But I am curious what @hawc2 and @hluling think about this. It would also imply another substantial rework of major parts of the tutorial.
Should I delete the section on "How Does Clustering Work?" I am open to doing so; just let me know what you think (it's becoming more and more difficult for me to have an "objective" view on my tutorial)
Thanks in advance for your help!
@thomjur @melaniewalsh @hawc2
Hey all, it sounds like most final revisions are clear and in agreement by all.
As far as walking through some introductory info about clustering using a toy (reduced) dataset, I think that's ok to do, so long as you explain what you've described in your response here, telling the reader why you are using a smaller dataset to explain certain concepts, and pointing out to the reader how this demo analysis differs slightly from the number of features you'll explore later.
I can understand why it might not be worth it to weave this part into the broader exploration of your main dataset, even if it would make the lesson a little more elegant, but it will be important at the very least that you give signposts so readers don't get led off track or get confused about how to connect different sections of the tutorial.
Thanks everyone for your time contributing to such a generative conversation about this important tutorial
Thanks for your follow-up comments, @hawc2 and @hluling.
Also @thomjur, to address your first question about whether or not to briefly show/discuss results, I still think it would be useful even though this isn't a research article because it will help show people how to interpret their own results. And I think that's part of what distinguishes Programming Historian from any other generalized scikit-learn tutorial on the internet. For a good model of briefly discussing results/interpretation, you might look at Matt Lavin's TF-IDF tutorial.
@hawc2 @melaniewalsh @hluling
I have now revised parts of my tutorial. Please let me know what you think. Note that some of the images do not seem to be displayed correctly (maybe it takes some time to refresh gh pages); also, the problem with the tables remains. It might make sense to wait a couple of days before you go through the tutorial again. I still have to adjust a few things. It's just that I had some spare time today and wanted to proceed with my tutorial.
Major issues
progressiveFeatureSelection( )
: I have added a couple of sentences to describe the overall idea of the algorithm. I have also tried to explain why I selected n=3 clusters (and why it might make sense to play with this parameter in the context of hyperparameter tuning).Minor Issues
Just a short update: The images should now be displayed correctly. I have also added a couple of sentences to the DBSCAN discussion and the summary related to @hluling claim to discuss/propose potential outcomes of the clustering.
@hluling @hawc2 @melaniewalsh
I have now revised my tutorial. In addition to the points mentioned above, I have also added some paragraphs to discuss potential further steps in the DSCAN section (see @hluling comments). Besides, I have also corrected some other mistakes; for instance, I realized that I was first talking about using n=3 clusters in the first part of the k-means clustering of the ancient author data and then suddenly changing to n=5 clusters in the actual analysis. I hope that I have now corrected everything to n=5.
Also, I was wondering about my use of "standardization" and "normalization." I think it's ok to use both terms since standardization is z-score normalization to my understanding, but maybe @hluling, as a trained statistician, could say some more about whether this use is correct, misleading, or simply wrong (I've made it through half a BA program in computer sciences by now, but I have not taken the - optional - class on statistics so far^^).
Thanks in advance for your help, and I am very much looking forward to your comments!
@thomjur Thanks for the updated version and some of the clarifications. I'm not a trained statistician, but I think the term 'standardization' better describes what you did based on para. 33. I think normalization typically means rescaling values to [0, 1].
Two more comments:
@hluling Thanks for your comments. I've replaced "normalized/normalization" with "standardized/standardization" in the tutorial and the Jupyter notebook. Thanks as well for the recommendation to mention scree plots (that's what you meant, I guess?). I've added a sentence to paragraph 100.
Regarding your second question: First of all, I tried running k-means with only a few clusters in the beginning. However, the elbow plot shows an almost linear line in this case, and the results when looking at the articles in each cluster are not very promising. If you want, you can try this out yourself in the Jupyter notebook (which is something I recommend doing throughout the entire tutorial anyway, because obviously, my choice of clusters might not be the perfect solution - which is nothing I claim anyway).
The clusters are too big and include too many diverse articles with such a low number of clusters. I am also mentioning this in my article when discussing the elbow plot: "But is it likely that a journal such as Religion that covers a vast spectrum of phenomena (which are all, of course, related to religion) only comprises a few thematic clusters? Probably not." (para. 103) I think that the "nature" of the data demands a more fine-grained solution because the articles are indeed diverse. This is why I run k-means with n=100, and I really think that the results are astonishingly good. I checked again with several different randomly selected clusters in my notebook, and I am honestly surprised how well this works (I am only showing three exemplary clusters in the text, since I think that this is enough to demonstrate the basic idea and encourage the reader to explore the data him- or herself). In addition, the rather small number of articles in each cluster allows checking the coherency of the clustering results more easily (again, there are obviously clusters that don't make much sense).
To be honest, I was not really sold when @melaniewalsh initially told me to use k-means with textual data because I thought that this is not the best way to analyze textual data. Yet, I am really surprised how well this worked. Is it perfect? Probably not. Could I sell this solution to a major company as a professional recommendation system? For sure not. But it delivers decent/coherent results and insights into the data, and it hopefully encourages the reader to try it out with his/her own (textual) data.
Now, when it comes to DBSCAN, I think we both agree (and I also say this in the tutorial several times) that the clustering, in this case, isn't really convincing. This is, among others, also because at least some of the clusters are too big. Yet, it is difficult to play around with the eps value until I have a similar outcome as the k-means algorithm because this is not how one would use DBSCAN. On the other hand, some of the few DBSCAN clusters actually show interesting results, which is why I recommend combining these results with the results provided by k-means, may it be as part of a recommender system or as an insight into the data as part of the exploratory data analysis. I mean, we are not restricted to only using one algorithm during the exploratory data analysis. We can apply as many as we like, look at the results (of which some are more and some are less indicative), and just combine the insights we gained from each in our final evaluation (in the way ensemble learning does for classification tasks by using different models and then combining their results).
I just read through the revised version of the tutorial, and I have some thoughts about the DBSCAN conversation. I also found some typos and small issues while re-reading, which I've noted below.
Major While I agree with @hluling that a better illustration of how DBSCAN deals with noise/outliers would be nice, I think I understand what you're saying @thomjur and that you're more interested in demonstrating that DBSCAN produces some interesting clusters, too, which we might think of as additional insights in our exploration.
Either way, upon re-reading, I noticed that it's hard to read the Religion article titles for each DBSCAN cluster, and that it's also hard to read the sample abstract when the Religion data is first introduced in paragraph 11 (the abstract simply says "In contemporary..." before it gets cut off).
In the DBSCAN section at the end, would it be possible to make the clusters into regular tables instead of code outputs so that they're easier to read? Something like...
df_abstracts_labeled[df_abstracts_labeled["cluster"] == 75]["title"] |
Religion Article Title | Cluster |
---|---|---|
Checking the heavenly ‘bank account of karma’: cognitive metaphors for karma in Western perception and early Theravāda Buddhism | 75 | |
Karma accounts: supplementary thoughts on Theravāda, Madhyamaka, theosophy, and Protestant Buddhism | 75 | |
Resonant paradigms in the study of religions and the emergence of Theravāda Buddhism | 75 |
Minor
@melaniewalsh Thanks a lot for your constructive comments! I've changed the code output to markdown tables in both the k-means and DBSCAN sections, and indeed, it looks much cleaner and is more readable. Great, thanks a lot! However, I left the abbreviated "In contemporary (...)" in the introductory part since this table only illustrates the structure of the table (otherwise, the table would be way too big/long when displaying the entire abstract). Yet, I am happy to change this if you would like me to. This also depends on the general question of how the tables will be displayed in the final tutorial. For instance, some of them currently include too many columns that are not shown when I open the draft web version in my browser. I am not sure whether there will be automatic breaks or something like this in the final template. Maybe @hawc2 knows more.
I've also corrected all the minor issues, except the quote by Patel . I personally like it, and I find it very helpful (plus, it makes my tutorial more "professional" when quoting someone who actually is an expert in this field), but I am also happy to delete it. I've also deleted the awkward film reference, which was, on top, wrong. -.-
Thanks again for your help, and have a great weekend!
@thomjur @hluling @melaniewalsh thank you all for this incredibly generative peer review process! It's amazing how much this tutorial has grown in complexity and detail over the last couple months. We have a few loose ends to tie up, but the big problems sound resolved. I'm recommending we now close the review and move on to the next stage of the publication process.
@thomjur I'll follow up with you about next steps. If you have any further chagnes to make, please go ahead and do so in the coming weeks.
@hluling and @melaniewalsh thank you so much for your investment in reviewing and improving this tutorial. It is a very difficult subject, especially to make user-friendly, and your expertise and attention to detail has really ensured it will be a popular Programming Historian lesson!
I the author Thomas Jurczyk hereby grant a non-exclusive license to ProgHist Ltd to allow The Programming Historian English to publish the tutorial in this ticket (including abstract, tables, figures, data, and supplemental material) under a CC-BY license.
@svmelton, this lesson is ready for next stages of the publication process. I’m providing the location of all relevant files, let me know if you need anything else.
lesson file - /lessons/clustering-with-scikit-learn-in-python.md
images folder - /images/clustering-with-scikit-learn-in-python/
assets folder - /assets/clustering-with-scikit-learn-in-python/
gallery image - /gallery/clustering-with-scikit-learn-in-python.png
gallery original - /gallery/originals/clustering-with-scikit-learn-in-python.png
The following author bio is for adding to the /_date/authors.yml file on the other repository:
- name: Thomas Jurczyk
team: false
orcid: 0000-0002-5943-2305
bio:
en: |
Thomas Jurczyk is a German historian and religious studies' scholar based in Bochum (GER)
Thank you @hawc2! This will be the first lesson that our new publishing assistant will work on for the copyediting! She's just started and I'll be meeting with her next week, and I'll let you know once the copyediting is complete and we're ready to publish.
Hi @Anisa-ProgHist! This is the lesson that's now ready for copyediting. Please let me know if you have any questions.
Thank you, @svmelton! I will take a look and be in touch if questions arise. Looking forward to working on this!
Hello @thomjur and @hawc2 ,
Thank you for your patience. My copyedits of this lesson are now ready for review.
I have applied my suggested revisions directly to the markdown file in our Submissions Repository, where you be able to view the additions and subtractions I’ve made in the Commit History
In addition, I will paste a list of my comments and suggested revisions here, so that the copyediting process is transparent and can invite discussion.
I have formatted my comments/suggested revisions as a task list. You will notice that many of these tasks are ‘checked’, because I have made the changes directly. A small number remain ‘unchecked’ because they ask questions.
I hope my comments will prove clear and useful. I’m happy to talk through anything which isn’t, and will not be offended if you choose not to absorb my suggestions.
--
A few general thoughts first, if I may.
--
[x] Para. 30. Italicise the k in k-means throughout
[x] Para. 30. Suggestion: add a link to Wikipedia definition k-means clustering
[x] Para. 30. Suggestion: add a link to Wikipedia definition DBSCAN
[x] Para. 30. Suggested revision: replace ‘among’. Instead: ’in combination with’
[x] Para. 30. Suggested revision: replace 'with', Instead 'using'
[x] Para. 32. Suggested revision: replace ‘the’ with ‘these two datasets’
[x] Para. 32. Suggested revision: replace ‘that’ with ‘how clustering algorithms’
[x] Para. 32. Suggested revision: replace ‘cover’ with ‘be applied to a broad range’
[x] Line 34. Suggested revision: replace ‘part’ with ‘section’
[x] Heading Line 36. Italicise Brill's New Pauly
[x] Para. 37. Suggested revision: replace ‘exemplary’ with ‘example’. I think it is rare to use exemplary as a noun in this way. Or is this an exemplary use case?
[x] Para. 37. Suggested revision: In this example, we will use k-means to analyze …
[x] Para. 37. Do you think it is important to say that access is not free? This access appears to be available for purchase (both for individuals or institutions)
It isn't clear until you read on, what the abbreviation DNP represents. Perhaps simpler to introduce the abbreviation after the first use of the original German title. Also, if you restructure this sentence, you can avoid the awkwardness of two apostrophes in Brill's New Pauly’s. Suggest: revision of sentence,
The data was taken from the official Brill's New Pauly website, and originates from Supplement I Volume 2: Dictionary of Greek and Latin Authors and Texts. Der Neue Pauly: Realenzyklopädie der Antike (1996–2002) is a well-known encyclopedia of the ancient world with contributions from established international scholars. The original German version has been translated into English since 2002. I will refer to the text using its German abbreviation (DNP) from here onwards.
[ ] Q: ’encyclopedia’ or ‘encyclopaedia’?
[x] Para. 39. Suggested revision: add comma after 'structure,' and replace ‘hypotheses that we already had’ with ‘existing hypotheses about the data.’ or simply, ‘existing hypotheses.’
[x] Para. 39. Suggested revision: replace ‘limited’ with ‘few’
[x] Para. 39. Suggested revision: replace ‘One the contrary’ with ‘Meanwhile’
[x] Para 39. The word 'may' is not needed in addition to the conditional ‘could’. Suggest delete ‘may’. ‘Another scenario could be that we find’
[x] Para. 39. Suggested revision: add 'associated' - 'groups of authors associated with many'
[x] Para. 39. Suggested revision: replace ‘still’ with ‘continue to rely’
[x] List, lines 45-52. Full stops / periods are not required after each list item
[x] Line 54. Suggested revision: replace ‘Consequently’ with ‘So, a single row …’
[x] Para. 62. I’m not sure if the word ‘posses’ is right here. They did not have scrape-able abstracts, or did not have abstracts at all? Suggested revision: However, some articles, particularly those published in older volumes, do not have abstracts, and thus were not scraped.
[ ] In your original sentence, Q: ‘scrapable’ or ‘scrape-able’?
[x] Para. 62. I’m not clear about the meaning of this sentence: ‘The same holds true for other contribution types such as reviews or miscellanea.’ What holds true? Their data was not scrape-able? Reviews and miscellanea have been left out of the dataset?
[x] List, lines 66-69. Full stops / periods are not required after each list item
[x] Line 71. Suggest: replace ‘Consequently’ with ‘So, a single row …’
[x] Line 79. Replace ‘with’ with ‘containing’ or ‘including’, […] a Jupyter notebook containing the code […]’
[x] Para 82. This paragraph briefly sets out the lesson's prerequisites of knowledge and understanding. However, the technological dependencies and operating system requirements are not defined anywhere in the lesson. Are K-means and DBSCAN suitable for use across Mac, Windows and Linux? Which environments has the lesson been developed and tested in?
[x] Heading Line 84. Italicise K-means
[x] Para. 85. Suggest replace ‘k-means’ with 'it': ‘which makes it an excellent model to start with,’
[x] Para. 85. Suggested revision: ‘Among others’ to ‘Among other capabilities’
[x] Para. 85. Suggested revision: ‘allows focusing on’ to ‘allows you to focus on’
[x] Para. 85. Suggest deleting ‘of’ to revise ‘while leaving noise points or outliers outside of the dense clusters’ to ‘while leaving noise points or outliers outside the dense clusters’
[x] Para. 85. Suggest addition: which is something that k-means cannot do independently
[x] Para. 87. Suggest addition: implementing other clustering algorithms in combination with scikit-learn […]
[x] Para. 87. Suggest revise ‘should be fairly easy’ to ‘should be fairly straight-forward’ to avoid repetition of easy/easily in this paragraph.
[x] Para. 87 Suggest choose between either ‘in general’ or ‘it is often’. Suggested revision: ’In general, it is advisable ...'
[x] Para. 90 Suggested revision: Machine learning is an artificial intelligence process by which computers can learn from data without being explicitly programmed
[x] Para. 90 Suggest add links: supervised, unsupervised, and reinforcement
[x] Para.92 Suggest add link: labeled data
[x] Para. 92. This is unclear: ‘predict already labeled data’ Suggested revision: ’One way to assess a supervised machine learning model's accuracy is to test it on some pre-labeled data, then compare the machine learning model's labeling predictions with the original output.
[x] Para. 92. Suggested revision: ‘Among others’ to ‘Among other things,’
[x] Para. 92. Suggested revision: ‘amount’ to ‘quantity’
[x] Para. 94. Suggested revision: ‘mostly works with unlabeled data.’ to ‘is applied to unlabeled data’
[x] Para. 94. Suggested revision: ‘Among others’ to ‘Among other things,’
[ ] Para. 94. Q: prelabeled or pre-labeled? appears twice in this paragraph
[x] Para. 94. Suggested revision: ’A good example are the datasets in our tutorial: We are only passing our model the author or abstract data,’ to ‘The datasets in this tutorial are a good example: we are only feeding our model either the author or abstract data,
[x] Para.99 Suggest add link: so-called naive k-means
[x] Para.99 This is unclear. Could you consider revising: ‘meaning that the cluster centers’ to ‘in which the cluster centers’
[ ] Para.99 Suggest add link to definition of ‘centroid’. I can’t find a straight-forward definition on Wikipedia, but I found this “A centroid is a data point (imaginary or real) at the center of a cluster” at https://www.fon.hum.uva.nl/praat/manual/k-means_clustering_1__How_does_k-means_clustering_work_.html Do you think this would be suitable?
[x] Para. 102. Suggested revision: ‘it is helpful to only consider very few features in this introductory part’ to ‘it is helpful to focus on a few key features in this introductory section’
[x] Para. 111. Suggested revision: ‘Since you mostly don't know how many clusters exist in your data’ to ‘In most cases, you will not know how many clusters exist in your data, so choosing the appropriate initial number of clusters is already a tricky question.’
[x] Para. 125. Suggested revision: ‘centroids are placed between’ to ‘centroids are positioned between’
[x] Para. 125. Suggested revision: ‘that we intuitively assume to represent one cluster’
[x] Para. 138. Suggested revision: ‘a change of words’ to ‘a change of word count’
[x] Para. 138. Suggested revision: ‘three authors who have all entries of approximately the same length in the DNP, but vary significantly concerning the number of their known works’ to ‘three authors who each have entries of approximately the same word count in the DNP, but who have a significantly different number of known published works’.
[x] Para. 138. Suggest: delete 'still'
[x] Para. 138. Suggested revision: the same number of documented works
[x] Para. 153. Suggested revision: ‘changing the number of words’ to ‘changing the word count’
[x] Para. 158. Suggested revision: There is no one-size-fits-all solution to this problem Q: unless, in this case, there is a one-size-fits-all solution?
[x] Para. 158. I intuit that ’n’ = number? If so, then it may be simpler to write: ‘right number of clusters for your data’, and in the following sentence ‘different numbers of clusters’
[x] Para. 158. Suggest add link: so-called elbow method
[ ] Para. 162. Q: should n
be written as code?
[ ] Para. 162. This is unclear: ‘The inertia decreases with the n number of clusters to the extreme that the inertia is zero when n is equal to the number of data points’. Can it be broken into two sentences?
Suggested revision: ‘The inertia decreases with the number of clusters. The extreme is that inertia will be zero when n is equal to the number of data points’.
[x] Para. 169. Suggested revision: ‘Another possibility to evaluate’ to ‘Another possible way to evaluate’
[x] Para. 173. Suggested revision: ‘Plotting the average silhouette score of all data points combined with the silhouette score of each data point in a cluster’ … to ‘Plotting the average silhouette score of all data points against the silhouette score of each data point in a cluster’
[x] Para. 173. Suggested revision: ‘can help to evaluate the model quality and the current choice of parameter values’ to ‘can help you to evaluate the quality of your model and the suitability of your current choice of parameter values’
[x] Para. 175. Q: do the words ‘dummy’ and ‘fictive’ need to be in brackets?
[x] Line 196. k-keams I think this is a typing error: k-means
[x] Para. 200. With accessibility in mind, I think it might be best to adjust ’vertical red line’ to ‘vertical dashed line’
[x] Para. 206. Suggested revision: ‘classifying data points without any direct neighbors as noise points’’ to ‘classifying data points without any direct neighbors as outliers or ‘noise points’.’
[x] Para. 206. Suggested revision: ‘include dense regions with data points’ to ‘include dense regions of data points’
[x] Para. 211. Suggested revision: ‘the minimum amount of data points that should be located in the neighborhood of data point to define a region as dense’ to ‘the minimum number of data points that should be located in the neighborhood of a data point to define its region as dense’
[x] Para. 212. Suggested revision: ‘minimum amount’ to ‘minimum number’
[x] Para. 213. Suggested revision: ‘If an initial cluster was found’ to ‘If an initial cluster is found’
[x] Para. 213. Suggested revision: ‘they are also included’ to ‘they will also be included’
[x] Para. 213. Suggested revision: ‘In case border points are part of different clusters, they are associated with the nearest cluster’ to ‘In cases where border points are part of different clusters, they will be associated with the nearest cluster’
[x] Para. 216. Suggested revision: ‘A helpful method to find the proper eps value is explained’ to ‘A helpful method for finding the proper eps value is explained’
[x] Para. 216. These sentences seem fragmentary and are unclear when the paragraph is read: ‘In short, we can plot the distance between each data point in a dataset and its nearest neighbor. We then sort the distance in ascending order. Finally, we look for the point in the plot with the steepest ascent (which allows a visual evaluation of the eps value, which is quite similar to the elbow method in the case of k-means)’.
How about:
‘In short, DBSCAN enables us to plot the distance between each data point in a dataset and identify its nearest neighbor. It is then possible to sort by distance in ascending order. Finally, we can look for the point in the plot which initiates the steepest ascent and make a visual evaluation of the eps value, similar to the ‘elbow’ evaluation method described above in the case of k-means)’.
[x] Heading Line 220. Suggested revision: Applying K-Means on the Ancient Authors Dataset from Brill's New Pauly to Applying K-Means to the Ancient Authors Dataset from Brill's New Pauly
[ ] Line 224. Q: Should pandas be italicised?
[ ] Para. 277. Q: Is Euclidean distance the same as ε-distance (eps) (Line 211)?
[x] Para. 279. Suggested revision: 75th percentile value and 90th percentile range
[x] Para. 365. Suggested revision: ‘The algorithm first looks for a’ to ‘The algorithm first identifies a’
[x] Para. 365. Q: What is a ‘feature’?
[x] Para. 415. Suggest add comma: ‘cast the mean of the columns to approximately zero and the standard deviation to one, to account for the huge differences between’
[x] Para. 425. Suggested revision: As we are only dealing with ten features, we could theoretically do this manually. However, because we have already implemented a basic algorithm to help us find potentially interesting combinations of features, we can also use our progressiveFeatureSelection()
function.
Replacing the word ‘since’ with ‘as’ and ‘because’
[x] Para. 425. Suggested revision: ‘was a random choice which unexpectedly led to some exciting results; however, this does not mean that there are no other promising combinations which might be worth examining.
[x] Para 459. Suggested revision: Yet, because the two other clusters are far below the average silhouette score for n=3 clusters, we decide to analyze the dataset with k-means using n=5 clusters.
[x] Para. 462. Suggested revision: I prefer plotting in two dimensions in Python, so we will use PCA()
(Principal Component Analysis) to reduce the dimensionality of our dataset to two dimensions.
Deleting the word ‘since’ and adding ‘so’
Deleting the word ‘since’ and adding comma + as
[x] Para 468. Suggest: deleting the word ‘however’ and beginning the sentence with ’42 is an arbitrary choice’. I feel that ‘however’ reads a little strangely when combined with ‘but’ in this sentence.
[x] Para 491. Suggested revision: ‘We were able to observe’ and ‘analyze whether their grouping’
[x] Para 493. Suggested revision: ‘only few known works’ to ‘very few known works’ and ‘it mostly consists of’ to ‘it largely consists of’
[x] Para. 508. Suggest revise this sentence: As we can see in this snippet showing the first ten entries in cluster 0, the authors' names (except Aesop) more or less support our initial hypothesis that we are mainly dealing with relatively unknown authors, particularly compared to the authors in cluster 4.
There are a lot of adverbs in this sentence. The combination of mainly, relatively and particularly make what is being said here quite vague. Can you be more specific?
relatively unknown authors = authors whose work is produced in few modern editions?
How about:
we are predominately dealing with authors whose work is produced in few modern editions, particularly compared to the authors in cluster 4.
[x] Para. 510. Suggested revision: The authors in cluster 4 (the less cohesive cluster at the upper right of our plot) comprise well-known and extensively discussed authors including Plato and Aristophanes, who have all written several works that are still famous and have remained relevant over the centuries, demonstrated by the high number of modern editions and commentaries.
[x] Para 526. Suggested revision: For instance, we could now take these clusters and apply our hypothesis about their relevance to further explore clustering the authors, based on their early and modern translations/editions. However, this is beyond the scope of this tutorial, which is primarily concerned with introducing tools and methods to examine such research questions.
[x] Para 529. Suggested revision: The second section of this tutorial will deal with textual data,
[x] Para 529. Suggest add link: TF-IDF
[x] Para 532. Suggested revision: Using a similar method as that used to analyze the DNP_ancient_authors.csv
dataset, we will first load the RELIGION_abstracts.csv
into our program and look at some summary statistics.
[x] Para. 550. I am still not clear on the definition of ‘features’…
[x] Para. 558. Suggest adding clarification of the verb ‘lemmitize’ because it is very unusual.
How about: As an optional step, I have implemented a function called lemmatizeAbstracts()
that groups, or ‘lemmatizes’ the abstracts
Q: features = words? Q: what is meant by ‘lemmatized version of the abstracts’? This is unclear.
[x] Para. 585. Suggest format the word argument
as code at each mention because it is a Reserved Word.
[x] Para. 596. Suggested revision: in at least five different documents but in no more than 200.
[x] Line 598. Suggest add link: Principal Component Analysis
[x] Line 598. Suggest add link: reduce the dimensionality
[x] Para. 609. Suggested revision: setting the dimensionality to d=10 was a random choice that happened to produce promising results. However, feel free to play around with these parameters while conducting a more elaborate hyperparameter tuning. Maybe you can find values for these parameters that result in an even more effective clustering of the data. For instance, you might want to use a scree plot to figure out the optimal number of principal components in PCA, which works quite similarly to our elbow method in the context of k-means.
[x] Para. 619. Suggested revision: Next, we try to find a reasonable method for clustering the abstracts using k-means. As we did in the case of the DNP_ancient_authors.csv
dataset, we will start by searching for the right number of clusters applying the elbow method and the silhouette score.
[x] Para. 623. Suggested revision: Therefore, let us continue by skipping the silhouette score plots (which are most likely of no value with such a huge number of clusters
[x] Para. 678. Suggest replace ‘albeit’ with ‘despite’
[x] Para 678. This is unclear. ‘able to assist us as a basic recommender system’ How about: ‘able to assist us by doing the work of a basic recommender system’
[x] Para. 678. Suggest add link: recommender system
[x] Para. 680. Suggested revision: Yet, as the textual data in this example is rather difficult to cluster and includes noise points or clusters that contain very few articles, it might make better sense to apply a different clustering algorithm and see how it performs.
[x] Para 686. Capitalise Euclidean
[x] Para 686. Suggested revision: If we were to use the initial TF-IDF matrix with 250 features, we would need to consider changing the underlying metric to cosine distance, which is more suitable when dealing with sparse matrices, as in the case of textual data.
[x] Para. 708. This is unclear. ‘fitting a DBSCAN instance under this circumstances’ ‘using a DBSCAN instance under these circumstances’?
[x] Para. 708. Word missing? ‘and a vast noise points cluster (-1) with more than 150 entries and an even bigger cluster with more than 500 entries’
[x] Para. 708. This is unclear. ‘As is also visible in the plot of these clusters in figure 12 (using a PCA-reduced dataset), this is not really helpful, and we should consider using the original TF-IDF matrix with cosine distance instead’. Could this be revised?
How about:
‘These clusters are plotted in figure 12 (using a PCA-reduced dataset), where the inconclusive results become even more visible. In this case, we could consider using the original TF-IDF matrix with cosine distance instead.
[x] Para. 710. Suggested revision: Its shortcomings aside, the current version of our DBSCAN instance does give some promising insights
[x] Para. 749. Suggested revision: Although the clustering was far from perfect in this case, it did produce some valuable information, which we could use in combination with the more promising results of the k-means clustering.
[x] Para. 749. Suggested revision: ‘Of course, we could also apply some other clustering algorithms and then combine the results’.
[x] Para. 751. Suggested revision: As a next step, we could pursue the idea of building a basic recommender system which suggests articles with similar topics to readers based on their previous readings.
[x] Para. 751. Suggested revision: When applied in combination, the rather unsatisfactory results of the DBSCAN model might be less problematic because they are now used as additional information only.
[x] Para. 753. Suggested revision: Of course, we as scholars in the humanities will be more likely to use these techniques as part of our research during the exploratory data analysis phase.
[x] Para. 753. Suggested revision: In this case, combining the results of different clustering algorithms helps us to discover structures and thematic clusters in our data.
[x] Para. 753. Suggested revision: an overview of research trends in the study of religion throughout recent decades.
[x] Bibliography Lines 762-764.
e.g.: https://www.chicagomanualofstyle.org/tools_citationguide/citation-guide-1.html Smith, Zadie. Swing Time. New York: Penguin Press, 2016.
Géron, Aurélien. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. Concepts, tools, and techniques to build intelligent systems, 2nd ed. Sebastopol: O’Reilly, 2019.
Mitchell, Ryan. Web scraping with Python. Collecting more data from the modern web, 1st ed. Sebastopol: O’Reilly, 2018.
Patel, Ankur A. Hands-on unsupervised learning using Python: How to build applied machine learning solutions from unlabeled data, 1st ed. Sebastopol: O’Reilly, 2019.
Hello @thomjur.
A couple of additional notes re: links. At the moment, you define a number of terms/link to a number of sources more than once in this lesson. Each technical term or resource only needs to be defined/linked to upon its first mention.
Let me know what you think.
Hi @Anisa-ProgHist ,
Thank you so much for all the work you put into copyediting this article! This looks great, and I agree with (almost) all your revisions.
First of all, let me respond to the first two issues you raised:
Concerning the "minor" issues you mentioned:
lemmatizeAbstracts()
function."?I have not changed/accepted anything yet, since I feel that there are still a few things that we need to discuss/agree on first. Afterward, I am happy to implement the changes myself if you would like me to. Just let me know whatever works best for you.
Again, thank you very much for your thorough revision of my tutorial! Great work!
All the best from Bochum, Thomas
Thank you for your swift and thoughtful replies, @thomjur.
@hawc2 and I are meeting next week, so perhaps he and I can talk through the points above, and come back to you with our thoughts on these remaining questions.
I appreciate your time on this, as well as your willingness to discuss the suggested adjustments.
Very best for now, Anisa
@thomjur it would be helpful to add a few more sentences describing how to get the code up and running in a Jupyter notebook.
Where you say, "You can download both datasets as well as a Jupyter notebook containing the code we are writing in this tutorial from this GitHub repository," you could tell the reader they can learn more about getting started by looking at this PH tutorial: https://programminghistorian.org/en/lessons/jupyter-notebooks.
Maybe also say something like "This lesson will work on any operating system, as long as you follow these instructions to set up an environment with Anaconda or Google Colab to run the Jupyter notebook locally or in the cloud."
Thank you, @hawc2. Hello there, @thomjur. Thank you for your patience.
A slight adjustment to your suggestion:
It should be noted that access to texts (and thus data) in the DNP is not free of charge. I was able to obtain the data discussed in this Case Study via my university’s institutional access provision. However, I have not copied any texts from the DNP to build this dataset. Numerical data used in the following analyses was extracted and partially accumulated from the author entries.
—
[x] 1. Let’s stick with ‘encyclopedia’ in para. 37, as this seems to be the standard American English spelling. This will be consistent with your other choices, analyze, initialized, normalize, etc.
[x] 2. Para. 62. I mentioned that I wasn’t sure about the choice of the word ‘posses’ here, because it is unclear whether you are saying that the articles did not have scrape-able abstracts, or did not have abstracts at all. I have suggested this revision: However, some articles, particularly those published in older volumes, do not have abstracts, and thus were not scraped. If you are happy with my suggestion, then we avoid my question of ‘scrapable' or 'scrape-able’. With your clarification, I think the final sentence of para.62 could be adjusted to read: Other contribution types, including reviews and miscellanea, have also been excluded from this dataset.
[x] 3. Para. 99. I’m pleased if you’re happy with the link I suggested to provide readers with a definition of ‘centroid’.
[x] 4. Para. 162. Alex and I talked about this and agree that n is not to be written as code, so we can leave that as it is. I'm pleased that you're happy with my suggestion to spilt the sentence about inertia into two shorter ones.
[x] 5. Para 175. We’ve agreed that the brackets around the words ‘dummy’ and ‘fictive’ can be removed.
[x] 6. Line 224. Alex and I talked about this too, and agree that pandas does not need to be italicised.
[x] 7. Para. 277. I think it sounds as though part of your explanation here ”according to the scikit-learn documentation (sklearn.cluster.DBSCAN — scikit-learn 0.24.2 documentation), they are using Euclidean distance as the default metric for the ε-distance. However, the DBSCAN "ε-distance" can also apply a different (non-Euclidean) metric” could be usefully incorporated into the text. As a (non-specialist) reader, I was unclear about whether these two were interchangeable.
[x] 8. Para. 365, let’s add a link to the Wikipedia definition of ‘feature’ in the context of machine learning, as you suggest.
[x] 9. Para. 558, my suggested revision was just intended to clarify the meaning of the verb ‘lemmitize’ which I have never heard before. Are you happy with my suggested addition of that groups, or ‘lemmatizes’ the abstracts? Otherwise, I think that sentence is fine and does not need changing.
[x] 10. Excellent. I’m pleased you agreed with us about the links. When the final adjustments are complete, I can go through and take out the repetitions, ensuring that links are associated with the first mention of a technical term/concept only.
If you are able to deal with the two general points 1 and 2 above, then I am happy to make the other agreed changes 1-10 below. Let me know if this sounds good to you.
Very best, Anisa
@hawc2 @Anisa-ProgHist
Thanks for the suggestions! I've uploaded a new version including few more changes. I left those issues that you both need to decide.
This lesson will work on any operating system, as long as you follow these instructions to set up an environment with Anaconda or Google Colab to run the Jupyter notebook locally or in the cloud. If you do not know how to set up a Jupyter notebook locally, this excellent PH tutorial might help you get started.
It should be noted that access to the texts (and thus the data) in the New Pauly is not free of charge. I used my university's access to obtain the data from the author entries. For the following analyses, I have not copied any texts from the New Pauly to the dataset. However, the numerical data in the dataset was extracted and partially accumulated from the author entries in the New Pauly. The original German version has been translated into English since 2002. I will refer to the text using its German abbreviation (DNP) from here onwards.
[x] Replaced "The same holds true for other contribution types such as reviews or miscellanea." with "I have also excluded other contribution types such as reviews or miscellanea from the dataset."
[x] Deleted brackets from "fictive" and "dummy."
[x] Added a link to the Wiki "Feauter (ML)" entry when the term is first mentioned.
[x] Changed ‘meaning that the cluster centers’ to ‘in which the cluster centers’
[x] Deleted the redundant links (however, I think that some of them are actually necessary, see the Wiki link to k-means in line 119. What do you think? I also added you changes to the links (for instance, linking dimenionality reduction in line 462)
Okay! Thank you, @thomjur!
I'll take another look.
Very best, Anisa
@Anisa-ProgHist Thanks! I think I'll leave it to you for the moment. I've adjusted the parts you asked for, I guess. however, you might still need to replace the part about the accessibility of the data with your revised version. But I'll stop working on the text for the moment, does not make much sense if we both work on the text at the same time. Thank you for all your help and let me know if there's anything else I can do!
Thank you for your work on this, @thomjur!
I have made some final small adjustments to the lesson file, which you will soon be able to review via the Commit History. The changes I've made include:
adding a link to the definition of TF-IDF at paragraph 77, and removing it from paragraph 529
removing the link to the definition of hyperparameter tuning in paragraph 94, as it is already available in paragraph 92
adding a link to the definition of centroids to paragraph 99, as discussed above
removing the link to the definition of elbow method at paragraph 158, as it is already available in paragraph 94
In paragraph 99, I'd like to suggest adding a link to the wikipedia definition of k-means++, rather than repeating the link to the Stanford article. I have adjusted the wording very slightly to reflect that: "I recommend reading this article by David Arthur and Sergei Vassilvitskii if you want to learn more."
You're absolutely right, in places the sentence structure means that it makes best sense to repeat a link, for example, as in paragraph 119 where you refer directly to a Wikipedia entry. This is also true at paragraph 169 "Silhouette (clustering)" where the link is repeated from 95, as well as at paragraph 209 where the DBSCAN wikipedia article link is repeated from paragraph 30.
On reflection, I think repeated links are also necessary in sentences where you are directing a reader to assets or data files associated with the lesson. Therefore, I have added back in the links to our GitHub repository at paragraph 79 and 524.
[ ] The only remaining aspect for you to consider relates to paragraph 277 and 211. Do you feel it could be useful to include an additional sentence to explain the relationship/difference between Euclidean distance and ε-distance? Or would you rather leave this as it is? (As I said, I am a non-specialist reader, and we have provided a link to the wikipedia definition of Euclidean distance in paragraph 115, so readers will have access to the information).
Almost there!
N.B.
Thank you for this, @Anisa-ProgHist! This looks great!
Regarding the remaining issue: I am not sure if we should make things too complex, but it might be helpful to, at least, mention the issue you raised by adding a short statement. I was thinking of something like:
The first step consists of defining an ε-distance (eps) that defines the neighborhood region (radius) of a data point. Just like in the case of k-means-clustering, scikit-learn's DBSCAN implementation uses Euclidean distance as the standard metric to calculate distances between data points.
What do you think, @Anisa-ProgHist and @hawc2?
Have a lovely weekend!
Thank you, @thomjur! I think that's helpful, and I have made this final change. @hawc2 has already provided the file directory locations, and shared your author bio in his comment above, so I think we are ready to hand this lesson over to @svmelton.
Great! I'll start working on this and let you know when it's been published.
We're live! Thanks to everyone for their great work on this. @hawc2 we're ready to add this to the twitter spreadsheet whenever you're good to go, let me know if you have questions.
Thank you all for the tremendous work you put into the publication of this paper. I am happy that it's out there!
Cheers, Thomas
@svmelton @anisa-hawes Just one question: the DOI https://doi.org/10.46430/phen0092 seems to point to the article "Text Reuse ..."; is this only a placeholder or did you assign the wrong DOI? Thanks!
Thank you, @thomjur! Apologies! @svmelton has already noted this, we are working together to resolve it today.
The Programming Historian has received the following tutorial on 'Unsupervised Learning and K-Means Clustering with Python' by @thomjur. This lesson is now under review and can be read at:
Old slug: https://programminghistorian.github.io/ph-submissions/lessons/k-means-clustering-with-scikit-learn-in-python
Final url for submission lesson: https://programminghistorian.github.io/ph-submissions/lessons/clustering-with-scikit-learn-in-python
Please feel free to use the line numbers provided on the preview if that helps with anchoring your comments, although you can structure your review as you see fit.
I will act as editor for the review process. My role is to solicit two reviews from the community and to manage the discussions, which should be held here on this forum. I have already read through the lesson and provided feedback, to which the author has responded.
Members of the wider community are also invited to offer constructive feedback which should post to this message thread, but they are asked to first read our Reviewer Guidelines (http://programminghistorian.org/reviewer-guidelines) and to adhere to our anti-harassment policy (below). We ask that all reviews stop after the second formal review has been submitted so that the author can focus on any revisions. I will make an announcement on this thread when that has occurred.
I will endeavor to keep the conversation open here on Github. If anyone feels the need to discuss anything privately, you are welcome to email me.
Our dedicated Ombudsperson is (Ian Milligan - http://programminghistorian.org/en/project-team). Please feel free to contact him at any time if you have concerns that you would like addressed by an impartial observer. Contacting the ombudsperson will have no impact on the outcome of any peer review.
Anti-Harassment Policy
This is a statement of the Programming Historian's principles and sets expectations for the tone and style of all correspondence between reviewers, authors, editors, and contributors to our public forums.