Technical review: Edge Cases: The Making of Network Navigator and Critical Approaches to DH Tools

[jdh-observer]

PID: nAW4E5etkMR6

https://github.com/jdh-observer/nAW4E5etkMR6

[inactinique]

sent first remarks to author(s):

• May I ask you to create an anonymized version of the notebook? You’re among the last authors who will have to do that, as, to speed up the technical review, we will switch to single blind peer review. But, as this issue has started with double-blind, I suggest carrying on with double blind;
• There is no cover image: would you find one, that would be pertinent to the article?
• The citations look fine: in fact, with some recent versions of jupyter notebook, there’s no display of cite2c quotes BUT everything is still in the notebook, so it works when you display it on the JDH website, such as here: https://journalofdigitalhistory.org/en/notebook-viewer/JTJGcHJveHktZ2l0aHVidXNlcmNvbnRlbnQlMkZqcmxhZGQlMkZlZGdlLWNhc2VzLUpESCUyRm1haW4lMkZhcnRpY2xlLXRleHQuaXB5bmI=?idx=15&layer=narrative&lh=1112&pidx=8&y=167
• Nevertheless, several of them seems to stick to the “.” that preceeds it. May I ask you to correct that? It might be a bit strange for reviewers.
• We want the altair graphs, because we want the graph interactivity o Note that on the preview on the JDH, the altair graphs will not be displayed: that is a known ‘bug’ (we cannot solve it for the preview: the notebook has to be trusted to display altair graphs, as altair is injecting javascript in python.
• Concerning figures, o Figure 1 has no title (in fact no metadata): can you add the metadata part as you did for the other ones? o The wordcloud is not tagged as a figure, and it should be o Same for the “popularity of network analysis tools…” graph o Same for the “ranking of the most popular network analysis tools” graph o History of network Navigator via Github o Popularity of the term infrastructure o Popularity of the term tool o Popularity of the term network o Popularity of the term network analysis o Tool criticism o Algorithm criticism o The nx.draw(G) cell
• To give a title to those graphs, you should edit the metadata of the cells. Here is an example:

"jdh": { "object": { "source": [ "Early ancestor of Silene stenophylla, recovered after being frozen for about 31,800 years" ] } } }

• Table o Please add the metadata and the tag to the “Data on the Six Degrees of Francis Bacon Quakers Group » table«

[inactinique]

Answer of the authors:

• Added a cover image (it’s an AI generated one which I explain in the article, but let us know if you would prefer a different image)
• Added figure tags (along with narrative and hermeneutics) to all listed figures.
• Edited all citations to be prior to the period and added relevant page numbers following the citation and within the parentheses.
• Added the requested table tag.
• Added source metadata to all cells. I did this programmatically using this script edge-cases-JDH/blob/main/script/cell_formatter.py and then saving the sources in a json file so if there are issues with our current formatting or content, please let us know and it should be relatively easy to change.
• Finally, I created two versions of the anonymized article because I wasn’t sure how blind you wanted us to go for this article since it’s a bit different than most academic articles where we are partially our own subject matter:
  • The first version is available here edge-cases-JDH/blob/main/article-text-partial-anonymous.ipynband; simply removes our names from the initial cells but leaves our names throughout the article
  • The second version fully anonymizes the article (available here edge-cases-JDH/blob/main/article-text-fully-anonymized.ipynb). To do this, I did a combination of manually work: removing our information from the initial cells (like ORCID links), redacting any link that included one of our names (which means we can’t link to the tool itself), and blocking out our names in any screenshot, and then programmatically replaced our names with Author1 or Author2 (or authors were relevant) and updated the cell metadata to also be anonymized (largely meant removing links).

Hopefully one of these versions works for the double-blind peer review and happy to answer any questions or make any additional adjustments.

Also thank you for sharing about the Altair charts, I was trying to get them to render in the Preview with no luck. We’re using Altair version 5 syntax throughout the article so hoping that doesn’t cause any issues on your end, though again please let us know and we’ll do our best to update the code.

[inactinique]

Analysing the fully anonimized version:

• the cover image does not display strangely (tag seems ok)
  • seems that the cell is just not run
• to be removed: title bibliography + cell with the insertion of bibliography
• figure 2: legend appears twice
  • probably because there's twice the figure, simply (once image, once code)
  • same figure 7, 12, 15, 16, 19
• figure 8 problem of anonymization (github link)
  • same figure 12, 13, 15, 16

[eliselavy]

@inactinique i will look at the reproducibility part

[inactinique]

Note that running the notebook on mybinder worked perfectly.

[eliselavy]

Using the https://jdh-binder.curvenote.dev/ instance: ERROR: No matching distribution found for altair==5.1.0

[inactinique]

Authors told me they had used altair 5. Is that a problem?

[eliselavy]

@inactinique i am clarify the libraries used as it seems

pip freeze

as been made, see list of libraries here https://github.com/jdh-observer/nAW4E5etkMR6/blob/main/requirements.txt

to include only useful

[eliselavy]

altair==5.0.1 ok

need to include:

nltk.download('punkt')
nltk.download('stopwords')

[eliselavy]

Generated from mybinder

Altair visualization generated see here: https://journalofdigitalhistory.org/en/notebook-viewer/JTJGcHJveHktZ2l0aHVidXNlcmNvbnRlbnQlMkZqZGgtb2JzZXJ2ZXIlMkZuQVc0RTVldGtNUjYlMkZyZXZpZXctMTQwJTJGYXJ0aWNsZS5pcHluYg==?idx=18&layer=narrative&lh=688&pidx=18&pl=hermeneutics&y=414

But problem table-1

=> need to download the notebook from mybinder and generate locally:

Table 1 Table of Example Six Degrees of Francis Bacon data Need to check one error bacon dataframe but only on my binder

see https://app.reviewnb.com/jdh-observer/nAW4E5etkMR6/pull/1/discussion/#comment-1900751151

Data on the Six Degrees of Francis Bacon Quakers Group:

Graph with 96 nodes and 162 edges
Density: 0.0355
Diameter: 8
Avg. Shortest Path Length: 3.3789
Transitivity 0.1695

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_35/2455966844.py in <module>
     31 # Convert node table to Pandas DataFrame for display
     32 nodes = pd.DataFrame.from_dict(G.nodes, orient='index')
---> 33 nodes.reset_index(level=0,names="name",inplace=True)
     34 nodes.sort_values("degree_centrality", ascending=False)

/srv/conda/envs/notebook/lib/python3.7/site-packages/pandas/util/_decorators.py in wrapper(*args, **kwargs)
    309                     stacklevel=stacklevel,
    310                 )
--> 311             return func(*args, **kwargs)
    312 
    313         return wrapper

TypeError: reset_index() got an unexpected keyword argument 'names'

But problem locally - nbconvert
Figure-3 Popularity of Network Analysis Tools Over Time and Across Multiple Datasets network_tools_over_time_chart ValidationError: 'legend' is not of type 'object'

Figure-5 History of Network Navigator via GitHub nn_chart ValidationError: 'legend' is not of type 'object'

Figure-9 Popularity of the Term ‘Infrastructure’ Over Time and Across Multiple Datasets infrastructure_tool_chart ValidationError: 'legend' is not of type 'object'

Figure 10 Popularity of the Term ‘Network Analysis’ Over Time and Across Multiple Datasets network_analysis_chart ValidationError: 'legend' is not of type 'object'

Figure 11 Popularity of the Term ‘Algorithmic Criticism’ Over Time and Across Multiple Datasets tool_algo_criticism_chart ValidationError: 'legend' is not of type 'object'

Remarks: notebook provided , addition trusted notebook, output are not correct

[eliselavy]

Perfect with data-table

[eliselavy]

Author's python version: 3.9.1 but runtime still set python-3.7

re-run code now with python version set as the author

  },
  "kernelspec": {
   "display_name": "edge-cases-venv",
   "language": "python",
   "name": "edge-cases-venv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  },

[eliselavy]

Reproducibility fixed with good version of python Will be added in the technical, check see here: https://github.com/C2DH/journal-of-digital-history-ipynb-preflight-action/issues/2

remove reference to the author 's repo only this reference , but example dataset: https://github.com/melaniewalsh/sample-social-network-datasets

Support of the data-table deployed in production: https://journalofdigitalhistory.org/en/notebook-viewer/JTJGcHJveHktZ2l0aHVidXNlcmNvbnRlbnQlMkZqZGgtb2JzZXJ2ZXIlMkZuQVc0RTVldGtNUjYlMkZtYWluJTJGYXJ0aWNsZS5pcHluYg==?idx=91

Sent to peer-review